adrien
83ff27c3f2
A bit better docs README
2025-01-12 01:41:42 +01:00
adrien
67546d69a9
Splited fileEngine into multiple files
2025-01-12 01:25:09 +01:00
adrien
955aff0d09
Moved global error to lib and fuse to a unique one
2025-01-12 00:37:57 +01:00
adrien
c0e8b07025
Moved thread stuff into a directory
2025-01-12 00:11:03 +01:00
adrien
d775ff1d0a
Deleted stuffs dir for a dataStructure one
2025-01-11 23:41:03 +01:00
adrien
bc0c0cbe1d
Update benchmark and some stuff
2025-01-11 19:26:06 +01:00
adrien
13caff2fd7
Dump csv
...
Can now use command dump csv
Will see later to add json and zid format as well as loading it
2025-01-11 19:25:03 +01:00
adrien
ee9d5c92f2
Some data struct now use file instead of = struct {
2025-01-11 19:24:15 +01:00
adrien
78213df3ff
And other
2025-01-11 18:03:36 +01:00
adrien
90edb94f7a
Same for schemaEngine
2025-01-11 17:58:52 +01:00
adrien
d5f7309869
Same for ziqlParser
2025-01-11 17:57:19 +01:00
adrien
0f6f34e706
Moved fileEngine to use the root as struct
2025-01-11 17:55:56 +01:00
adrien
bd4f0aab7f
Removed some TODO, fix and typo
2025-01-11 16:52:19 +01:00
adrien
54146b9495
Add and remove UUID from index map
...
Now remove and add UUID in the sstruct map UUID -> File index
2025-01-11 16:08:46 +01:00
adrien
1495e779c9
Speed up date
...
Date was taking a long time for parsing when using ADD in batch, speeded
up by like x50
2025-01-11 15:27:17 +01:00
adrien
7d012b527f
Write every 1000 map
...
Now if I run a big ADD query, write every 1000 entities to prevent a
maps array too big and go OOM
2025-01-11 13:50:32 +01:00
adrien
5300f932d5
Update README
2025-01-08 23:53:56 +01:00
adrien
d533eaff98
Working multi struct schema with relationship!
...
Noce
2025-01-08 23:53:41 +01:00
adrien
71e5f6eb1e
Started to debug schema with multiple struct and some time keyword
...
Added NOW already and now debuging some stuff regarding filter and
parsing file of one struct when it should be another
Also moved query test into a seperated test file.
And some fix and changed in docs
2025-01-08 10:09:15 +01:00
adrien
a712fe3e83
Save schema file when using new
2025-01-07 17:15:09 +01:00
adrien
65fa7ccfc4
Now delete empty file when deleting entities exept 0.zid
2025-01-07 14:09:22 +01:00
adrien
2a4842432d
Speed up batch ADD and better bechmark
...
Now I flush only when the file is full and I check the the currently
used file if it is big enough.
So I dont get stat of all files and flush everytime like before
2025-01-07 13:55:02 +01:00
adrien
e3264d8553
Random ADD for benchmark
2025-01-06 20:45:16 +00:00
adrien
900b9e5305
Moved benchmark outside of src and making it better
2025-01-02 13:28:23 +00:00
adrien
b075f8b89a
Moved config to libs
2025-01-02 12:19:05 +00:00
adrien
4f421f7977
Added ADD in batch
2024-12-30 15:31:16 +01:00
adrien
a535ce5283
Basic benchmark command
2024-12-30 14:09:59 +01:00
adrien
a7d076ec80
Added !IN
2024-12-30 12:08:25 +01:00
adrien
2e2dae3df1
Some typo
2024-12-29 10:19:04 +01:00
adrien
b2dd6fe627
Basic working relationship
...
It is working !
The basic GRAB [name, best_friend] work and do return the best friend
infos.
I added more test and some do not work yet, working on it now
2024-12-28 20:12:20 +01:00
adrien
3b47007ca4
Base reparsing
...
This dosnt work yet but I implemented the reparsing of the files to
return relationship.
So GRAB User [name, friends] should return all infos of all friends and
not just the UUID of those friends like before
2024-12-27 12:42:51 +01:00
adrien
0127daa330
Push before leaving for christmas
2024-12-21 15:13:50 +01:00
adrien
e7056efec9
Now query with relationship will write the UUID bytes betwen {|<>|}
...
So then I can parse the file again, create a map of UUID sub json and
just iterate over the first json looking for UUID and update with new
data from map
2024-12-20 22:29:02 +01:00
adrien
7fb0b5f63e
Changed memory in AdditionalData
...
Now it need to use an arena because there is no deinit function.
And additionalData of member are null by default instead of an empty
list.
2024-11-30 10:33:28 +01:00
adrien
57a9eecdb3
Renamed members of AdditionalData
2024-11-30 09:49:54 +01:00
adrien
c901ec2b06
Added some docs
2024-11-29 21:57:04 +01:00
adrien
c6fe428270
Dont send link if not in AdditionalData
2024-11-29 21:56:52 +01:00
adrien
4ee79c9629
Created an EntityWriter responsable for writing into JSON and table into a writer
2024-11-29 21:38:05 +01:00
adrien
7f27557ca2
Fix parsing array
...
Before if the array is [ 1 ] I get [ 0, 1, 0] because I get 2 empty
string that when I try to make into int, it become 0
2024-11-29 21:20:12 +01:00
adrien
ae2fa848b4
Added docs
2024-11-28 19:21:09 +01:00
adrien
3f53da5220
Making link_array working, now can do query with filter for array of ofther struct
2024-11-28 19:20:44 +01:00
adrien
12ae7f9a62
Added db state command and fixed printError
...
I was using the same buffer at 2 place :/
2024-11-26 22:59:36 +01:00
adrien
3c01150e33
Update Docs
2024-11-26 21:31:53 +01:00
adrien
e41f53b692
Update commands docs
2024-11-26 20:10:22 +01:00
adrien
5141df3429
Added pygments in ci workslow
2024-11-26 19:49:57 +01:00
adrien
481fee27bf
Added code highlit
2024-11-26 19:43:02 +01:00
adrien
874550a77d
Added git repo to docs
2024-11-26 19:37:57 +01:00
adrien
2e3e89d3db
Update docs
2024-11-26 19:34:44 +01:00
adrien
75a8e822a1
Test material theme
2024-11-26 19:17:29 +01:00
adrien
9c460b5ab4
Update docs
2024-11-26 19:14:22 +01:00
adrien
7e5abcedea
Update Docs
...
Shoud added them on docs and added images
2024-11-26 18:59:12 +01:00
adrien
cc32eff1da
Added doc website complete as Roadmap
2024-11-26 07:19:52 +01:00
adrien
c35b559a60
Update docs
2024-11-25 23:40:42 +01:00
adrien
a40670a0c6
Update to trigger github action
2024-11-25 23:35:43 +01:00
adrien
ed43a4b0ce
Update url in mkdocs
2024-11-25 23:31:01 +01:00
adrien
acef72b319
Delete Dockerfile
...
It is useless, it was just a quick test
2024-11-25 23:25:19 +01:00
adrien
fcc0cd71f2
Update ci
2024-11-25 23:24:23 +01:00
adrien
95f23449d6
Update Docs
2024-11-25 23:09:02 +01:00
adrien
401b40bb8f
Added glightbox
2024-11-25 22:49:59 +01:00
adrien
b54c25fdb8
Fix
2024-11-25 22:48:10 +01:00
adrien
2338cb6549
Added docs
2024-11-25 22:43:03 +01:00
adrien
08cae48cbc
Added a writeEntityTable and fixed date and time
2024-11-24 22:26:54 +01:00
adrien
1333b75981
Dont trow and error on basic relationship query
2024-11-24 20:10:21 +01:00
adrien
bf1f5916d7
Can now ADD using a filter
2024-11-24 16:55:54 +01:00
adrien
ceab51da27
NewData map now use ConditionValue
2024-11-24 16:26:26 +01:00
adrien
23e8ed8709
Changed a bit how the memory work
...
Now SchemaEngine have an arena at the root of the file. So UUIDFileIndex
can be as big as we want. The idea is nice, maybe I will use it for the
FileEngine too.
2024-11-22 11:03:39 +01:00
adrien
3f5c929a11
Big memory improvement
...
Started to use fixed buffer at the root of each file. I like that
because that mean I can estimate the amount of memory that the app use.
The only thing that still use an non fixed allocator is the
UUIDFileIndex, but I dont like it. I think it should also use a fixed
lenght buffer. But then how can I be sure it is enough ? :/
I mean I get it, I need an allocator here. But if this is the only place
I use one, this a shame
2024-11-20 23:52:36 +01:00
adrien
979690fec4
IN filter working
...
Now if I do like GRAB User { best_friends IN {name = 'Bob'}} it should
work
At least now the condition value is a hashmap with key as UUID.
2024-11-17 21:52:11 +01:00
adrien
7b2a754d81
Check if additional data count is reached at each iteration
...
Before I was checking after the filter.evaluate is true. So I can end up
with the limit reach, but the thread will continue parsing until it find
one that is true, add it and then check if the limit is reach.
Which mean I endup with more than what I want and the thread could stop
before, making things faster.
2024-11-17 20:45:58 +01:00
adrien
65ba0263cc
Condition value now can have a HashMap of UUID/void
2024-11-17 18:11:54 +01:00
adrien
ca5de646d6
Populate the UUIDFileIndex
...
Now each SchemaStruct have a UUIDFileIndex that allow to find the file
index of where an entity is based on it's UUID
2024-11-17 16:40:12 +01:00
adrien
5c8d2b6dc7
Create ThreadEngine
...
Now the DBEngine have a ThreadEngine that have a pool. Like that it is
always the same pool and I dont create one everytime I run a parseing
methods of the FileEngine
2024-11-17 15:25:54 +01:00
adrien
303dc81e2a
Created an unused UUIDFileIndex
...
Created a RadixTree but it is too slow and doest provide that much
reduced memory, mostly because I did it badly. So I will juste use a
small wrapper around an AutoHashMap UUID usize
2024-11-17 00:11:34 +01:00
adrien
f1c2bea5c1
Update Docs
2024-11-14 22:32:22 +01:00
adrien
ef84200b34
Parsing filter can now parse sub filter, waiting to be use with the parsing to find relationship
2024-11-14 20:39:04 +01:00
adrien
6e7d1d150c
Pass test with added stuffs for relationship
2024-11-13 22:44:27 +01:00
adrien
13cba58dbc
Fix
2024-11-12 21:57:13 +01:00
adrien
5e9c64caf6
Removed useless code
2024-11-12 21:18:19 +01:00
adrien
3a61e81024
Added not in keyword
2024-11-12 21:17:49 +01:00
adrien
b1de4a40c3
Moved ZipponData to lib
2024-11-12 21:17:33 +01:00
adrien
f5d93c94f6
Created a ThreadSyncContext
...
This keep the atomic value that are shared by thread
2024-11-12 21:17:06 +01:00
adrien
268010fe37
Removed log in testing for now
2024-11-10 17:10:56 +01:00
adrien
91b1f61e66
Working new SchemaEngine
2024-11-10 17:10:42 +01:00
adrien
dd65f5f03e
Typo in docs
2024-11-10 11:41:06 +01:00
adrien
e1c957b3b6
Started cleaning by creating a SchemaEngine
2024-11-10 11:38:37 +01:00
adrien
1d365adff6
Moved charts inside the python dir
2024-11-09 16:42:21 +01:00
adrien
18c55194d5
Moved stuffs to new dir
2024-11-09 16:41:28 +01:00
adrien
9fa6f25459
Removed CSV_DELIMITER and some old comment
2024-11-09 16:39:40 +01:00
adrien
e5d0a122e1
Added multi threading to delete
2024-11-06 16:04:07 +01:00
adrien
65be757440
Added multithreading to update
2024-11-06 16:01:55 +01:00
adrien
fbbeb2f40d
Updqte Benchmqrk.md
2024-11-04 23:24:49 +01:00
adrien
48a3ae4a5d
Update Benchmark.md
2024-11-04 23:20:24 +01:00
adrien
900b6c5f54
Replace some path_buff to utils.preintOpenDir/File
2024-11-04 23:18:49 +01:00
adrien
96e77a5ad4
Removed last trace of null
2024-11-04 22:57:19 +01:00
adrien
280b3b3c3a
Update docs
2024-11-04 22:52:44 +01:00
adrien
b12116c005
Removed uuid_file_index
2024-11-04 22:52:19 +01:00
adrien
463df1dc6c
Added time for 1 000 000
2024-11-04 22:38:17 +01:00
adrien
3cf33fe4f8
Fixed url of chart
2024-11-04 22:13:33 +01:00
adrien
2af9f68c5e
Added small benchmark
2024-11-04 22:11:56 +01:00
adrien
c500c2a23c
Fixed wrong condition
...
Superior was inferior and reverse
2024-11-03 22:43:02 +01:00
adrien