Now SchemaEngine have an arena at the root of the file. So UUIDFileIndex
can be as big as we want. The idea is nice, maybe I will use it for the
FileEngine too.
Started to use fixed buffer at the root of each file. I like that
because that mean I can estimate the amount of memory that the app use.
The only thing that still use an non fixed allocator is the
UUIDFileIndex, but I dont like it. I think it should also use a fixed
lenght buffer. But then how can I be sure it is enough ? :/
I mean I get it, I need an allocator here. But if this is the only place
I use one, this a shame
Before I was checking after the filter.evaluate is true. So I can end up
with the limit reach, but the thread will continue parsing until it find
one that is true, add it and then check if the limit is reach.
Which mean I endup with more than what I want and the thread could stop
before, making things faster.
Now the DBEngine have a ThreadEngine that have a pool. Like that it is
always the same pool and I dont create one everytime I run a parseing
methods of the FileEngine
Created a RadixTree but it is too slow and doest provide that much
reduced memory, mostly because I did it badly. So I will juste use a
small wrapper around an AutoHashMap UUID usize
- Added a new data type self, that represent the id of the intity itself
- Fixed multi threading for parsing, now each thread use it's own writer
and I concat them at the end
- Added a schemaStruct id to the list
- Other fixe and stuff to go with the rest
New step, multi threading for all function then finally relationship
For very importatnt stuff like the writter that write data when parsing,
started to use fixed length because it take the majority of the time to
write, not to parse =/
Gonna need to improve that
When I use s2t, or string to type. Like to transform a string array into
an array. "[1 4 21]" to []i32{ 1, 4, 21}.
Before I return an ArrayList, now it use toOwnedSlice
Created a new function to replace parseAndFindUUIDWithFilter.
Now it parse the file, evaluate with the filter and write directly to a
buffer. There is no UUID that is return, then file are parse again.
Should be a huge perf improve.
Some bugs with additionData tho, got a name: "<UUID>" =(
SchemaStruct member now are [] and not ArrayList. Started to use and
understand toOwnedSlice.
Implemented the hashmap that keep all UUID -> File index. Inside each
SchemaStruct
Condition now store the index of the value in the file instead of the
member_name. So when I parse, I can just use the list array []Data and
the index to compare both value
Now the parseFilter function return a Filter struct.
It is a tree of Condition and Logic operator that will then be use when
parsing row. Need to do the evaluate function with the []Data and test
some error
Now log are managed by the std log, did a custom log to print either
into a file or into the default io strout.
The plan is to create a scope in each file and log depending of stuff,
can also use it in library I think, if should still base by the custom
function.
Next step is logging a lot of stuff a bit everywhere.
Added a hash map that contain the name of the struct corresponding to
the link.
So the member "best_friend: User" is an entry key: "best_friend" value:
"User"
Before it was link that and I changed to use Loc as I throught that I
would save memory.
But a slice is just a pointer and a leng, so it should be better. As
long as I keep the original string in memory. The query and schema
Added the datatype date, time and datetime
Moved all custom erros to a single file
Cleaned code to be more readable and created some small utils
Other stuffs that I dont recall. Basically preparing for the
relationship implementation
Removed the loop for the CLI, now just take some argument using the
binary. May come back to while loop if I need to keep like the file
engine between session. To run query in parallel.
Moved printError and send to utils and removed duplicate of them.
Organized FileEngine better.
Put the function that convert string to value in a seperate file.
Fix some synthax and make it smaller. Removed unused functions too
adrien