adrien/ZipponDB
1
0
Fork 0
Code Actions Releases Activity
262 Commits 2 Branches 0 Tags
15ec304fe1c2c00e0ddbb2da977ce932b2f4b05c
Commit Graph
157 Commits
Author SHA1 Message Date
adrien 15ec304fe1 APPEND work, need to check next array manipulation 2025-01-27 20:38:03 +01:00
adrien baf6cd2284 Added some log and started testing array manip. Fixed some errors already 2025-01-25 21:15:04 +01:00
adrien 78e8b67b80 Moved benchmark and test to src. Implemented array without testing yet 2025-01-25 19:11:01 +01:00
adrien 6fa8d6b7c6 Opti 
Bring test back

Removed useless arena, making like 40% perf improv

Removed shared Loc
 2025-01-23 17:31:14 +01:00
adrien 98f0c69e61 SOLVED THREAD BUG 
Needed a thread safe alloc. Will need to update other part as I just did
parseEntities and deleteEntities
 2025-01-22 22:34:46 +01:00
adrien f05696a924 Removed max_file from SyncContext to use work group 
Now instead of create a context with the number of file to parse and
check every x millisecond if it is complete, I use work group. For each
file, I spawn a thread in the group and then I call wg.wait()
 2025-01-22 18:24:02 +01:00
adrien 7c34431702 Error found!! 
Got an error that I could find for a while. It was when parsing a lot of
files.

Turn out it was the Thread Pool or something because if I run on 1 core,
it's ok.

Different thing in this commit, I just want to freeze a state that
benchmark work.
 2025-01-22 11:34:00 +01:00
adrien ebf91bb61c If config.CPU_CORE == 0, use the cpu count of processor 2025-01-22 10:29:54 +01:00
adrien abe9445d72 Compilable array manipulation, still some errors 2025-01-20 19:56:29 +01:00
adrien 14ea22dad6 Added a map with just the zero UUID in memory 
So I dont need to create a map everytime I have a orders=none
 2025-01-16 22:49:36 +01:00
adrien 6bd3782e1e Now string with \n are written in triple quote 2025-01-16 22:34:39 +01:00
adrien a09a368528 Removed useless code and old comment 2025-01-16 22:34:19 +01:00
adrien d1b430a3d5 Removed some TODO and changer so now it parse all existing file 
Before I was parsing all files, getting the max index then parsing from
0 to the max.

Byt now that I delete empty files, I need to parse only existing one.
 2025-01-16 22:17:42 +01:00
adrien 154a146678 Missing from previous commit 2025-01-16 20:45:50 +01:00
adrien a9c39d9aa5 Renamed paarser.core to ziql.parser 2025-01-16 20:45:30 +01:00
adrien 6338cb6364 NewData ordered by default 2025-01-15 19:59:13 +01:00
adrien e848be94c1 Good docs update, stillo need some work. Need to take a step back and think 2025-01-14 22:48:50 +01:00
adrien 9ae9b65f1e Rename ziql to parser 2025-01-14 22:42:11 +01:00
adrien 173b302976 Add number of file to db metrics 2025-01-14 22:31:55 +01:00
adrien e818700905 Use try 2025-01-13 20:06:16 +01:00
adrien c6725f6b54 Fix stdout error 2025-01-12 20:28:35 +01:00
adrien c070723baa ziql moved into a dir and splited 2025-01-12 15:33:15 +01:00
adrien 1f5da7a74b Created cli struct and dir 2025-01-12 12:37:07 +01:00
adrien 1c8384dec6 Moved stuff into a schema dir and created empty dir 2025-01-12 11:50:22 +01:00
adrien 67546d69a9 Splited fileEngine into multiple files 2025-01-12 01:25:09 +01:00
adrien 955aff0d09 Moved global error to lib and fuse to a unique one 2025-01-12 00:37:57 +01:00
adrien c0e8b07025 Moved thread stuff into a directory 2025-01-12 00:11:03 +01:00
adrien d775ff1d0a Deleted stuffs dir for a dataStructure one 2025-01-11 23:41:03 +01:00
adrien 13caff2fd7 Dump csv 
Can now use command dump csv

Will see later to add json and zid format as well as loading it
 2025-01-11 19:25:03 +01:00
adrien ee9d5c92f2 Some data struct now use file instead of = struct { 2025-01-11 19:24:15 +01:00
adrien 78213df3ff And other 2025-01-11 18:03:36 +01:00
adrien 90edb94f7a Same for schemaEngine 2025-01-11 17:58:52 +01:00
adrien d5f7309869 Same for ziqlParser 2025-01-11 17:57:19 +01:00
adrien 0f6f34e706 Moved fileEngine to use the root as struct 2025-01-11 17:55:56 +01:00
adrien bd4f0aab7f Removed some TODO, fix and typo 2025-01-11 16:52:19 +01:00
adrien 54146b9495 Add and remove UUID from index map 
Now remove and add UUID in the sstruct map UUID -> File index
 2025-01-11 16:08:46 +01:00
adrien 1495e779c9 Speed up date 
Date was taking a long time for parsing when using ADD in batch, speeded
up by like x50
 2025-01-11 15:27:17 +01:00
adrien 7d012b527f Write every 1000 map 
Now if I run a big ADD query, write every 1000 entities to prevent a
maps array too big and go OOM
 2025-01-11 13:50:32 +01:00
adrien d533eaff98 Working multi struct schema with relationship! 
Noce
 2025-01-08 23:53:41 +01:00
adrien 71e5f6eb1e Started to debug schema with multiple struct and some time keyword 
Added NOW already and now debuging some stuff regarding filter and
parsing file of one struct when it should be another

Also moved query test into a seperated test file.
And some fix and changed in docs
 2025-01-08 10:09:15 +01:00
adrien a712fe3e83 Save schema file when using new 2025-01-07 17:15:09 +01:00
adrien 65fa7ccfc4 Now delete empty file when deleting entities exept 0.zid 2025-01-07 14:09:22 +01:00
adrien 2a4842432d Speed up batch ADD and better bechmark 
Now I flush only when the file is full and I check the the currently
used file if it is big enough.

So I dont get stat of all files and flush everytime like before
 2025-01-07 13:55:02 +01:00
adrien e3264d8553 Random ADD for benchmark 2025-01-06 20:45:16 +00:00
adrien 900b9e5305 Moved benchmark outside of src and making it better 2025-01-02 13:28:23 +00:00
adrien b075f8b89a Moved config to libs 2025-01-02 12:19:05 +00:00
adrien 4f421f7977 Added ADD in batch 2024-12-30 15:31:16 +01:00
adrien a535ce5283 Basic benchmark command 2024-12-30 14:09:59 +01:00
adrien a7d076ec80 Added !IN 2024-12-30 12:08:25 +01:00
adrien 2e2dae3df1 Some typo 2024-12-29 10:19:04 +01:00