15 Commits

Author SHA1 Message Date
71e5f6eb1e Started to debug schema with multiple struct and some time keyword
Added NOW already and now debuging some stuff regarding filter and
parsing file of one struct when it should be another

Also moved query test into a seperated test file.
And some fix and changed in docs
2025-01-08 10:09:15 +01:00
b075f8b89a Moved config to libs 2025-01-02 12:19:05 +00:00
3b47007ca4 Base reparsing
This dosnt work yet but I implemented the reparsing of the files to
return relationship.

So GRAB User [name, friends] should return all infos of all friends and
not just the UUID of those friends like before
2024-12-27 12:42:51 +01:00
ceab51da27 NewData map now use ConditionValue 2024-11-24 16:26:26 +01:00
23e8ed8709 Changed a bit how the memory work
Now SchemaEngine have an arena at the root of the file. So UUIDFileIndex
can be as big as we want. The idea is nice, maybe I will use it for the
FileEngine too.
2024-11-22 11:03:39 +01:00
3f5c929a11 Big memory improvement
Started to use fixed buffer at the root of each file. I like that
because that mean I can estimate the amount of memory that the app use.

The only thing that still use an non fixed allocator is the
UUIDFileIndex, but I dont like it. I think it should also use a fixed
lenght buffer. But then how can I be sure it is enough ? :/

I mean I get it, I need an allocator here. But if this is the only place
I use one, this a shame
2024-11-20 23:52:36 +01:00
7b2a754d81 Check if additional data count is reached at each iteration
Before I was checking after the filter.evaluate is true. So I can end up
with the limit reach, but the thread will continue parsing until it find
one that is true, add it and then check if the limit is reach.

Which mean I endup with more than what I want and the thread could stop
before, making things faster.
2024-11-17 20:45:58 +01:00
65ba0263cc Condition value now can have a HashMap of UUID/void 2024-11-17 18:11:54 +01:00
ca5de646d6 Populate the UUIDFileIndex
Now each SchemaStruct have a UUIDFileIndex that allow to find the file
index of where an entity is based on it's UUID
2024-11-17 16:40:12 +01:00
91b1f61e66 Working new SchemaEngine 2024-11-10 17:10:42 +01:00
e1c957b3b6 Started cleaning by creating a SchemaEngine 2024-11-10 11:38:37 +01:00
e2e8bc4d80 Created the engines folder
Created a new folder to clean a bit the repo, put the file and schema
engine inside. As those and Parser depend on the types.zig, I also add
this folder inside the new engines folder
2024-10-09 23:32:37 +02:00
b008f434a6 Passed to one tabular file for each struct
Created a new Parser unique for the FileEngine to read each line.
It is slower as I need to parser character by character because their is
no fixed len for the data in files. Before I was just reading until the
end of the file.

Im gonna need to find some tricks to improve the parsing of data. I am
thinking using the stream directly instead of doing streamUntilDelimiter
2024-10-09 23:20:28 +02:00
67fb49ded5 Working new ZiQL pqrser for basic ADD and GRAB !!! 2024-10-08 00:18:25 +02:00
44e48a5276 Big rework - Now use global ziql parser - still buggy, need to debug the tests 2024-10-07 00:40:24 +02:00