Now SchemaEngine have an arena at the root of the file. So UUIDFileIndex
can be as big as we want. The idea is nice, maybe I will use it for the
FileEngine too.
Started to use fixed buffer at the root of each file. I like that
because that mean I can estimate the amount of memory that the app use.
The only thing that still use an non fixed allocator is the
UUIDFileIndex, but I dont like it. I think it should also use a fixed
lenght buffer. But then how can I be sure it is enough ? :/
I mean I get it, I need an allocator here. But if this is the only place
I use one, this a shame
Before I was checking after the filter.evaluate is true. So I can end up
with the limit reach, but the thread will continue parsing until it find
one that is true, add it and then check if the limit is reach.
Which mean I endup with more than what I want and the thread could stop
before, making things faster.
Now the DBEngine have a ThreadEngine that have a pool. Like that it is
always the same pool and I dont create one everytime I run a parseing
methods of the FileEngine
Created a RadixTree but it is too slow and doest provide that much
reduced memory, mostly because I did it badly. So I will juste use a
small wrapper around an AutoHashMap UUID usize
- Added a new data type self, that represent the id of the intity itself
- Fixed multi threading for parsing, now each thread use it's own writer
and I concat them at the end
- Added a schemaStruct id to the list
- Other fixe and stuff to go with the rest
New step, multi threading for all function then finally relationship
For very importatnt stuff like the writter that write data when parsing,
started to use fixed length because it take the majority of the time to
write, not to parse =/
Gonna need to improve that
When I use s2t, or string to type. Like to transform a string array into
an array. "[1 4 21]" to []i32{ 1, 4, 21}.
Before I return an ArrayList, now it use toOwnedSlice
Created a new function to replace parseAndFindUUIDWithFilter.
Now it parse the file, evaluate with the filter and write directly to a
buffer. There is no UUID that is return, then file are parse again.
Should be a huge perf improve.
Some bugs with additionData tho, got a name: "<UUID>" =(
SchemaStruct member now are [] and not ArrayList. Started to use and
understand toOwnedSlice.
Implemented the hashmap that keep all UUID -> File index. Inside each
SchemaStruct
