adrien/ZipponDB
1
0
Fork 0
Code Actions Releases Activity
255 Commits 2 Branches 0 Tags
6fa8d6b7c6990227880b6a47f51328f23578add8
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
adrien 6fa8d6b7c6 Opti 
Bring test back

Removed useless arena, making like 40% perf improv

Removed shared Loc
 2025-01-23 17:31:14 +01:00
adrien 98f0c69e61 SOLVED THREAD BUG 
Needed a thread safe alloc. Will need to update other part as I just did
parseEntities and deleteEntities
 2025-01-22 22:34:46 +01:00
adrien f05696a924 Removed max_file from SyncContext to use work group 
Now instead of create a context with the number of file to parse and
check every x millisecond if it is complete, I use work group. For each
file, I spawn a thread in the group and then I call wg.wait()
 2025-01-22 18:24:02 +01:00
adrien 7c34431702 Error found!! 
Got an error that I could find for a while. It was when parsing a lot of
files.

Turn out it was the Thread Pool or something because if I run on 1 core,
it's ok.

Different thing in this commit, I just want to freeze a state that
benchmark work.
 2025-01-22 11:34:00 +01:00
adrien ebf91bb61c If config.CPU_CORE == 0, use the cpu count of processor 2025-01-22 10:29:54 +01:00
adrien 1ca2fc5470 Updated docs and roadmap 2025-01-20 19:56:44 +01:00
adrien abe9445d72 Compilable array manipulation, still some errors 2025-01-20 19:56:29 +01:00
adrien 14ea22dad6 Added a map with just the zero UUID in memory 
So I dont need to create a map everytime I have a orders=none
 2025-01-16 22:49:36 +01:00
adrien 6bd3782e1e Now string with \n are written in triple quote 2025-01-16 22:34:39 +01:00
adrien a09a368528 Removed useless code and old comment 2025-01-16 22:34:19 +01:00
adrien d1b430a3d5 Removed some TODO and changer so now it parse all existing file 
Before I was parsing all files, getting the max index then parsing from
0 to the max.

Byt now that I delete empty files, I need to parse only existing one.
 2025-01-16 22:17:42 +01:00
adrien a16c9b29c1 Update README and doc 2025-01-16 20:45:58 +01:00
adrien 154a146678 Missing from previous commit 2025-01-16 20:45:50 +01:00
adrien a9c39d9aa5 Renamed paarser.core to ziql.parser 2025-01-16 20:45:30 +01:00
adrien 6338cb6364 NewData ordered by default 2025-01-15 19:59:13 +01:00
adrien e848be94c1 Good docs update, stillo need some work. Need to take a step back and think 2025-01-14 22:48:50 +01:00
adrien 9ae9b65f1e Rename ziql to parser 2025-01-14 22:42:11 +01:00
adrien dff7ead5b1 . 2025-01-14 22:33:01 +01:00
adrien 26bbe0d0f9 . 2025-01-14 22:32:42 +01:00
adrien 672d79cbea Add config and schema for benchmark 2025-01-14 22:32:31 +01:00
adrien 173b302976 Add number of file to db metrics 2025-01-14 22:31:55 +01:00
adrien 77e1197894 Custom color for docs 2025-01-14 15:15:26 +01:00
adrien 425401d764 Change image and doc name 2025-01-14 13:43:48 +01:00
adrien 3a6a0e202b Update image 2025-01-14 13:42:22 +01:00
adrien 58bbc44119 Add transparent logo 2025-01-14 10:58:48 +01:00
adrien a5e957c05a Release for every tag now 2025-01-14 10:54:32 +01:00
adrien e818700905 Use try 2025-01-13 20:06:16 +01:00
adrien 1031e767bc Deleted logo dir and moved new logo to docs images 2025-01-13 20:04:05 +01:00
adrien bfb7533ef9 Merge branch 'main' into v0.1.8 2025-01-13 20:01:50 +01:00
adrien eb0389464d Update logo 2025-01-13 20:01:22 +01:00
adrien e18a12aefe Add simple SQL comparison 2025-01-13 10:56:53 +00:00
adrien 57f97e50ab Remove extension from zip name 2025-01-12 20:35:43 +01:00
adrien c6725f6b54 Fix stdout error 2025-01-12 20:28:35 +01:00
adrien 6ccfc7feb9 Send back to true and remove .pdb from release 2025-01-12 19:08:34 +01:00
adrien 35ca253dad Test 2025-01-12 18:32:25 +01:00
adrien d35b7a5e3e Fix ? 2025-01-12 18:27:12 +01:00
adrien c7024d70c4 Update release flow 2025-01-12 18:21:52 +01:00
adrien bd4cc98797 Release test 2025-01-12 17:16:16 +01:00
adrien 0b871ecc93 Add permision 2025-01-12 17:05:40 +01:00
adrien 374d724bba Update workflow 2025-01-12 16:52:26 +01:00
adrien d4d03f1b2c Typo 2025-01-12 16:42:38 +01:00
adrien f1a37648d3 Typo 2025-01-12 16:41:05 +01:00
adrien 574c616929 Rename workflow and remove arm in build release 2025-01-12 16:39:07 +01:00
adrien 7db5c5329a Update workflow 2025-01-12 16:34:59 +01:00
adrien 9b547e594e Update github action 2025-01-12 16:28:02 +01:00
adrien ff92cacc7c Auto release github action 2025-01-12 16:24:18 +01:00
adrien c070723baa ziql moved into a dir and splited 2025-01-12 15:33:15 +01:00
adrien 1f5da7a74b Created cli struct and dir 2025-01-12 12:37:07 +01:00
adrien 1c8384dec6 Moved stuff into a schema dir and created empty dir 2025-01-12 11:50:22 +01:00
adrien 83ff27c3f2 A bit better docs README 2025-01-12 01:41:42 +01:00
adrien 67546d69a9 Splited fileEngine into multiple files 2025-01-12 01:25:09 +01:00
adrien 955aff0d09 Moved global error to lib and fuse to a unique one 2025-01-12 00:37:57 +01:00
adrien c0e8b07025 Moved thread stuff into a directory 2025-01-12 00:11:03 +01:00
adrien d775ff1d0a Deleted stuffs dir for a dataStructure one 2025-01-11 23:41:03 +01:00
adrien bc0c0cbe1d Update benchmark and some stuff 2025-01-11 19:26:06 +01:00
adrien 13caff2fd7 Dump csv 
Can now use command dump csv

Will see later to add json and zid format as well as loading it
 2025-01-11 19:25:03 +01:00
adrien ee9d5c92f2 Some data struct now use file instead of = struct { 2025-01-11 19:24:15 +01:00
adrien 78213df3ff And other 2025-01-11 18:03:36 +01:00
adrien 90edb94f7a Same for schemaEngine 2025-01-11 17:58:52 +01:00
adrien d5f7309869 Same for ziqlParser 2025-01-11 17:57:19 +01:00
adrien 0f6f34e706 Moved fileEngine to use the root as struct 2025-01-11 17:55:56 +01:00
adrien bd4f0aab7f Removed some TODO, fix and typo 2025-01-11 16:52:19 +01:00
adrien 54146b9495 Add and remove UUID from index map 
Now remove and add UUID in the sstruct map UUID -> File index
 2025-01-11 16:08:46 +01:00
adrien 1495e779c9 Speed up date 
Date was taking a long time for parsing when using ADD in batch, speeded
up by like x50
 2025-01-11 15:27:17 +01:00
adrien 7d012b527f Write every 1000 map 
Now if I run a big ADD query, write every 1000 entities to prevent a
maps array too big and go OOM
 2025-01-11 13:50:32 +01:00
adrien 5300f932d5 Update README 2025-01-08 23:53:56 +01:00
adrien d533eaff98 Working multi struct schema with relationship! 
Noce
 2025-01-08 23:53:41 +01:00
adrien 71e5f6eb1e Started to debug schema with multiple struct and some time keyword 
Added NOW already and now debuging some stuff regarding filter and
parsing file of one struct when it should be another

Also moved query test into a seperated test file.
And some fix and changed in docs
 2025-01-08 10:09:15 +01:00
adrien a712fe3e83 Save schema file when using new 2025-01-07 17:15:09 +01:00
adrien 65fa7ccfc4 Now delete empty file when deleting entities exept 0.zid 2025-01-07 14:09:22 +01:00
adrien 2a4842432d Speed up batch ADD and better bechmark 
Now I flush only when the file is full and I check the the currently
used file if it is big enough.

So I dont get stat of all files and flush everytime like before
 2025-01-07 13:55:02 +01:00
adrien e3264d8553 Random ADD for benchmark 2025-01-06 20:45:16 +00:00
adrien 900b9e5305 Moved benchmark outside of src and making it better 2025-01-02 13:28:23 +00:00
adrien b075f8b89a Moved config to libs 2025-01-02 12:19:05 +00:00
adrien 4f421f7977 Added ADD in batch 2024-12-30 15:31:16 +01:00
adrien a535ce5283 Basic benchmark command 2024-12-30 14:09:59 +01:00
adrien a7d076ec80 Added !IN 2024-12-30 12:08:25 +01:00
adrien 2e2dae3df1 Some typo 2024-12-29 10:19:04 +01:00
adrien b2dd6fe627 Basic working relationship 
It is working !

The basic GRAB [name, best_friend] work and do return the best friend
infos.

I added more test and some do not work yet, working on it now
 2024-12-28 20:12:20 +01:00
adrien 3b47007ca4 Base reparsing 
This dosnt work yet but I implemented the reparsing of the files to
return relationship.

So GRAB User [name, friends] should return all infos of all friends and
not just the UUID of those friends like before
 2024-12-27 12:42:51 +01:00
adrien 0127daa330 Push before leaving for christmas 2024-12-21 15:13:50 +01:00
adrien e7056efec9 Now query with relationship will write the UUID bytes betwen {|<>|} 
So then I can parse the file again, create a map of UUID sub json and
just iterate over the first json looking for UUID and update with new
data from map
 2024-12-20 22:29:02 +01:00
adrien 7fb0b5f63e Changed memory in AdditionalData 
Now it need to use an arena because there is no deinit function.

And additionalData of member are null by default instead of an empty
list.
 2024-11-30 10:33:28 +01:00
adrien 57a9eecdb3 Renamed members of AdditionalData 2024-11-30 09:49:54 +01:00
adrien c901ec2b06 Added some docs 2024-11-29 21:57:04 +01:00
adrien c6fe428270 Dont send link if not in AdditionalData 2024-11-29 21:56:52 +01:00
adrien 4ee79c9629 Created an EntityWriter responsable for writing into JSON and table into a writer 2024-11-29 21:38:05 +01:00
adrien 7f27557ca2 Fix parsing array 
Before if the array is [ 1 ] I get [ 0, 1, 0] because I get 2 empty
string that when I try to make into int, it become 0
 2024-11-29 21:20:12 +01:00
adrien ae2fa848b4 Added docs 2024-11-28 19:21:09 +01:00
adrien 3f53da5220 Making link_array working, now can do query with filter for array of ofther struct 2024-11-28 19:20:44 +01:00
adrien 12ae7f9a62 Added db state command and fixed printError 
I was using the same buffer at 2 place :/
 2024-11-26 22:59:36 +01:00
adrien 3c01150e33 Update Docs 2024-11-26 21:31:53 +01:00
adrien e41f53b692 Update commands docs 2024-11-26 20:10:22 +01:00
adrien 5141df3429 Added pygments in ci workslow 2024-11-26 19:49:57 +01:00
adrien 481fee27bf Added code highlit 2024-11-26 19:43:02 +01:00
adrien 874550a77d Added git repo to docs 2024-11-26 19:37:57 +01:00
adrien 2e3e89d3db Update docs 2024-11-26 19:34:44 +01:00
adrien 75a8e822a1 Test material theme 2024-11-26 19:17:29 +01:00
adrien 9c460b5ab4 Update docs 2024-11-26 19:14:22 +01:00
adrien 7e5abcedea Update Docs 
Shoud added them on docs and added images
 2024-11-26 18:59:12 +01:00