Benchmark are set to evolve. I have currently multiple ideas to improve performance.
+ZipponDB is fairly fast and can easely query millions of entities. +Current limitation is around 5GB I would say, depending of CPU cores, usage and kind of data saved. +After that query can start to become slow as optimizations are still missing.
+Most of query's time is writing entities into a JSON format. Parsing the file itself take little time. +For example in the benchmark report bellow, I parse 100 000 users in around 40ms if there is no entities to send and 130ms if all 100 000 entities are to send.
+I choosed to release ZipponDB binary with the small release. +Zig has a fast and safe release, but the fast release isn't that much faster in my case, if not at all. +If you want you can build it with it.
+You can run zig build benchmark, if you clone the repo to benchmark your machine.
+More info on how to build from source.
Here an example on my machine with 16 core:
+=====================================
+
+Populating with 5000 users.
+Populate duration: 0.035698 seconds
+
+Database path: benchmarkDB
+Total size: 0.36Mb
+CPU core: 16
+Max file size: 5.00Mb
+LOG: 0.02Mb
+BACKUP: 0.00Mb
+DATA: 0.33Mb
+ Item: 0.00Mb | 19 entities | 1 files
+ User: 0.33Mb | 5000 entities | 1 files
+ Order: 0.00Mb | 0 entities | 1 files
+ Category: 0.00Mb | 4 entities | 1 files
+
+--------------------------------------
+
+Query: GRAB User {}
+Time: 16.90 ± 25.22 ms | Min 8.25ms | Max 92.55ms
+
+Query: GRAB User {name='asd'}
+Time: 2.62 ± 0.10 ms | Min 2.52ms | Max 2.85ms
+
+Query: GRAB User [1] {}
+Time: 0.16 ± 0.01 ms | Min 0.15ms | Max 0.18ms
+
+Query: GRAB User [name] {}
+Time: 7.88 ± 11.69 ms | Min 3.91ms | Max 42.94ms
+
+Query: GRAB User {name = 'Charlie'}
+Time: 3.87 ± 0.16 ms | Min 3.70ms | Max 4.17ms
+
+Query: GRAB Category {}
+Time: 0.20 ± 0.07 ms | Min 0.17ms | Max 0.41ms
+
+Query: GRAB Item {}
+Time: 0.21 ± 0.02 ms | Min 0.19ms | Max 0.25ms
+
+Query: GRAB Order {}
+Time: 0.14 ± 0.01 ms | Min 0.13ms | Max 0.18ms
+
+Query: GRAB Order [from, items, quantity, at] {}
+Time: 4.18 ± 12.01 ms | Min 0.15ms | Max 40.21ms
+
+Query: DELETE User {}
+Time: 0.64 ± 1.13 ms | Min 0.23ms | Max 4.04ms
+
+Read: 1907698 Entity/second *Include small condition
+Write: 350200 Entity/second
+
+=====================================
+
+Populating with 100000 users.
+Populate duration: 0.707605 seconds
+
+Database path: benchmarkDB
+Total size: 6.62Mb
+CPU core: 16
+Max file size: 5.00Mb
+LOG: 0.02Mb
+BACKUP: 0.00Mb
+DATA: 6.59Mb
+ Item: 0.00Mb | 19 entities | 1 files
+ User: 6.59Mb | 100000 entities | 2 files
+ Order: 0.00Mb | 0 entities | 1 files
+ Category: 0.00Mb | 4 entities | 1 files
+
+--------------------------------------
+
+Query: GRAB User {}
+Time: 126.99 ± 3.05 ms | Min 123.37ms | Max 133.56ms
+
+Query: GRAB User {name='asd'}
+Time: 38.12 ± 1.60 ms | Min 36.48ms | Max 41.88ms
+
+Query: GRAB User [1] {}
+Time: 0.19 ± 0.02 ms | Min 0.16ms | Max 0.22ms
+
+Query: GRAB User [name] {}
+Time: 59.33 ± 1.29 ms | Min 58.02ms | Max 61.47ms
+
+Query: GRAB User {name = 'Charlie'}
+Time: 53.29 ± 1.00 ms | Min 51.50ms | Max 54.78ms
+
+Query: GRAB Category {}
+Time: 0.19 ± 0.01 ms | Min 0.18ms | Max 0.22ms
+
+Query: GRAB Item {}
+Time: 5.51 ± 13.43 ms | Min 0.22ms | Max 45.22ms
+
+Query: GRAB Order {}
+Time: 0.16 ± 0.01 ms | Min 0.15ms | Max 0.18ms
+
+Query: GRAB Order [from, items, quantity, at] {}
+Time: 0.17 ± 0.02 ms | Min 0.15ms | Max 0.21ms
+
+Query: DELETE User {}
+Time: 5.96 ± 17.04 ms | Min 0.26ms | Max 57.07ms
+
+Read: 2623338 Entity/second *Include small condition
+Write: 1125278 Entity/second
+
+=====================================
+
+Populating with 1000000 users.
+Populate duration: 7.029142 seconds
+
+Database path: benchmarkDB
+Total size: 65.96Mb
+CPU core: 16
+Max file size: 5.00Mb
+LOG: 0.02Mb
+BACKUP: 0.00Mb
+DATA: 65.93Mb
+ Item: 0.00Mb | 19 entities | 1 files
+ User: 65.93Mb | 1000000 entities | 14 files
+ Order: 0.00Mb | 0 entities | 1 files
+ Category: 0.00Mb | 4 entities | 1 files
+
+--------------------------------------
+
+Query: GRAB User {}
+Time: 250.77 ± 6.74 ms | Min 247.08ms | Max 270.61ms
+
+Query: GRAB User {name='asd'}
+Time: 67.90 ± 0.42 ms | Min 67.31ms | Max 68.78ms
+
+Query: GRAB User [1] {}
+Time: 8.92 ± 24.86 ms | Min 0.55ms | Max 83.51ms
+
+Query: GRAB User [name] {}
+Time: 110.08 ± 5.27 ms | Min 106.86ms | Max 125.21ms
+
+Query: GRAB User {name = 'Charlie'}
+Time: 73.65 ± 2.79 ms | Min 69.24ms | Max 79.22ms
+
+Query: GRAB Category {}
+Time: 0.19 ± 0.04 ms | Min 0.16ms | Max 0.33ms
+
+Query: GRAB Item {}
+Time: 0.21 ± 0.02 ms | Min 0.19ms | Max 0.26ms
+
+Query: GRAB Order {}
+Time: 0.15 ± 0.01 ms | Min 0.14ms | Max 0.17ms
+
+Query: GRAB Order [from, items, quantity, at] {}
+Time: 0.17 ± 0.01 ms | Min 0.16ms | Max 0.18ms
+
+Query: DELETE User {}
+Time: 11.74 ± 34.19 ms | Min 0.29ms | Max 114.30ms
+
+Read: 14727354 Entity/second *Include small condition
+Write: 5468517 Entity/second
+
+=====================================
+
+Populating with 10000000 users.
+Populate duration: 72.675680 seconds
+
+Database path: benchmarkDB
+Total size: 659.33Mb
+CPU core: 16
+Max file size: 5.00Mb
+LOG: 0.02Mb
+BACKUP: 0.00Mb
+DATA: 659.30Mb
+ Item: 0.00Mb | 19 entities | 1 files
+ User: 659.30Mb | 10000000 entities | 132 files
+ Order: 0.00Mb | 0 entities | 1 files
+ Category: 0.00Mb | 4 entities | 1 files
+
+--------------------------------------
+
+Query: GRAB User {}
+Time: 2535.29 ± 86.92 ms | Min 2448.39ms | Max 2712.78ms
+
+Query: GRAB User {name='asd'}
+Time: 684.75 ± 39.96 ms | Min 649.09ms | Max 797.13ms
+
+Query: GRAB User [1] {}
+Time: 6.65 ± 1.00 ms | Min 5.36ms | Max 8.75ms
+
+Query: GRAB User [name] {}
+Time: 1106.21 ± 33.57 ms | Min 1056.57ms | Max 1172.61ms
+
+Query: GRAB User {name = 'Charlie'}
+Time: 690.56 ± 20.41 ms | Min 661.51ms | Max 718.07ms
+
+Query: GRAB Category {}
+Time: 0.21 ± 0.03 ms | Min 0.18ms | Max 0.31ms
+
+Query: GRAB Item {}
+Time: 0.23 ± 0.04 ms | Min 0.19ms | Max 0.32ms
+
+Query: GRAB Order {}
+Time: 0.15 ± 0.01 ms | Min 0.13ms | Max 0.17ms
+
+Query: GRAB Order [from, items, quantity, at] {}
+Time: 0.17 ± 0.02 ms | Min 0.15ms | Max 0.21ms
+
+Query: DELETE User {}
+Time: 109.55 ± 326.64ms | Min 0.47ms | Max 1089.46ms
+
+Read: 14603810 Entity/second *Include small condition
+Write: 5403847 Entity/second
+=====================================
+ZipponDB have a little set of types. This is on purpose, to keep the database simple and fast. But more type may be added in the future.
+ZipponDB supports 8 primary data types:
+| Type | +Description | +Example | +
|---|---|---|
| int | +32-bit integer | +42 | +
| float | +64-bit float (must include a decimal point) | +3.14 | +
| bool | +Boolean value | +true or false | +
| string | +Character array enclosed in single quotes | +'Hello, World!' | +
| UUID | +Universally Unique Identifier | +123e4567-e89b-12d3-a456-426614174000 | +
| date | +Date in yyyy/mm/dd format | +2024/10/19 | +
| time | +Time in hh:mm:ss.mmmm format | +12:45:00.0000 | +
| datetime | +Combined date and time | +2024/10/19-12:45:00.0000 | +
Any of these data types can be used as an array by prefixing it with []. For example:
[]int: An array of integers[]string: An array of stringsThis guide will help you set up and start using ZipponDB quickly.
+Obtain a binary for your architecture by:
+Once with the binary, run it to get access to the CLI.
+Once in the CLI, create a database by running: +
+Alternatively, set the ZIPPONDB_PATH environment variable.
Define a schema and attach it to the database by running:
+ +This will create the necessary directories and empty files for data storage. Test the current database schema by running:
+ +Alternatively, set the ZIPPONDB_SCHEMA environment variable.
Start using the database by sending queries, such as:
+ + + + + + + + + + + + + + + +Note: This will probably evolve over time.
+In ZipponDB, data is organized and manipulated using structures, or structs, rather than traditional tables. A struct is defined by a name, such as User, and members, such as name and age.
To declare structs for use in your database, create a separate file containing the schema definitions. Below is an example of a simple schema file: +
+In this example, the best_friend member is a reference to another User.
Here's a more complex example featuring multiple structs: +
User (
+ name: str,
+ email: str,
+ friends: []User,
+ posts: []Post,
+ comments: []Comment,
+)
+
+Post (
+ title: str,
+ image: str,
+ at: date,
+ like_by: []User,
+ comments: []Comment,
+)
+
+Comment (
+ content: str,
+ at: date,
+ like_by: []User,
+)
+Note: The [] symbol preceding a type indicates an array of that type. For example, []User represents an array of User structs.
+In future releases, ZipponDB will support schema updates, allowing you to modify existing structs or add new ones, and then apply these changes to your database. Currently, schema modifications are not possible once the database has been initialized.
+I will add commands or query to add, delete and update struct in the schema. Maybe the schema will be a Struct itself with a schema like that:
+ +Like that can just do ZiQL query directly on it.
+In the future I will migrate into a single file database like SQLite.
+A blok is like memory but for file. It has a starting point and a size and can hold entities or FileVar. +Everything I will save will be linked into a Blok and I can create and delete them at will. The size is in the config and can't change.
+In SQLite, it is what they call it a page I think. Similare thing.
+A FileVar is a single value but saved in a file. This have a Blok, a position, a size and a type. +For string, I will create something different I think, with a size max and a len, and can request more if needed.
+ + + + + + + + + + + + + +This could be fun, make a small demo where you get a wasm that run the database locally in the browser.
+ + + + + + + + + + + + + +TODO
+Note: Code snippets in this documentation are simplified examples and may not represent the actual codebase.
+Tokenizers are responsible for converting a buffer string into a list of tokens. Each token has a Tag enum that represents its type, such as equal for the = symbol, and a Loc struct with start and end indices that represent its position in the buffer.
All tokenizers work similarly and are based on the zig tokenizer. They have two main methods: next, which returns the next token, and getTokenSlice, which returns the slice of the buffer that represents the token.
+Here's an example of how to use a tokenizer: +
const toker = Tokenizer.init(buff);
+const token = toker.next();
+std.debug.print("{s}", .{toker.getTokenSlice(token)});
+Tokenizers are often used in a loop until the end tag is reached. In each iteration, the next token is retrieved and processed based on its tag. Here's a simple example:
+
const toker = Tokenizer.init(buff);
+var token = toker.next();
+while (token.tag != .end) : (token = toker.next()) switch (token.tag) {
+ .equal => std.debug.print("{s}", .{toker.getTokenSlice(token)}),
+ else => {},
+}
+There are four different tokenizers in ZipponDB:
+Each tokenizer has its own set of tags and parsing rules, but they all work similarly.
+Parsers are the next step after tokenization. They take tokens and perform actions or raise errors. There are three parsers in ZipponDB: one for ZiQL, one for schema files, and one for CLI commands.
+A parser has a State enum and a Tokenizer instance as members, and a parse method that processes tokens until the end state is reached.
Here's an example of how a parser works: +
var state = .start;
+var token = self.toker.next();
+while (state != .end) : (token = self.toker.next()) switch (state) {
+ .start => switch (token.tag) {
+ .identifier => self.addStruct(token),
+ else => printError("Error: Expected a struct name.", token),
+ },
+ else => {},
+}
+The parser's state is updated based on the combination of the current state and token tag. This process continues until the end state is reached.
The ZiQL parser uses different methods for parsing:
+parse: The main parsing method that calls other methods.parseFilter: Creates a filter tree from the query.parseCondition: Creates a condition from a part of the query.parseAdditionalData: Populates additional data from the query.parseNewData: Returns a string map with key-value pairs from the query.parseOption: Not implemented yet.File parsing is done through a small library that I did named ZipponData.
+It is minimal and fast, it can parse 1_000_000 entity in 0.3s on one thread +on a 7 7800X3D at around 4.5GHz with a Samsung SSD 980 PRO 2TB (up to 7,000/5,100MB/s for read/write speed).
+To read a file, you create an iterator for a single file and then you can iterate with .next(). It will return an array of Data. This make everything very easy to use.
const std = @import("std");
+
+pub fn main() !void {
+ const allocator = std.testing.allocator;
+
+ // 0. Make a temporary directory
+ try std.fs.cwd().makeDir("tmp");
+ const dir = try std.fs.cwd().openDir("tmp", .{});
+
+ // 1. Create a file
+ try createFile("test", dir);
+
+ // 2. Create some Data
+ const data = [_]Data{
+ Data.initInt(1),
+ Data.initFloat(3.14159),
+ Data.initInt(-5),
+ Data.initStr("Hello world"),
+ Data.initBool(true),
+ Data.initUnix(2021),
+ };
+
+ // 3. Create a DataWriter
+ var dwriter = try DataWriter.init("test", dir);
+ defer dwriter.deinit(); // This just close the file
+
+ // 4. Write some data
+ try dwriter.write(&data);
+ try dwriter.write(&data);
+ try dwriter.flush(); // Dont forget to flush !
+
+ // 5. Create a schema
+ // A schema is how the iterator will parse the file.
+ // If you are wrong here, it will return wrong/random data
+ // And most likely an error when iterating in the while loop
+ const schema = &[_]DType{
+ .Int,
+ .Float,
+ .Int,
+ .Str,
+ .Bool,
+ .Unix,
+ };
+
+ // 6. Create a DataIterator
+ var iter = try DataIterator.init(allocator, "test", dir, schema);
+ defer iter.deinit();
+
+ // 7. Iterate over data
+ while (try iter.next()) |row| {
+ std.debug.print("Row: {any}\n", .{ row });
+ }
+
+ // 8. Delete the file (Optional ofc)
+ try deleteFile("test", dir);
+ try std.fs.cwd().deleteDir("tmp");
+}
+ZipponDB segregate responsability with Engines.
+For example the FileEngine is the only place where files used, for both writting and reading. This simplify refactoring, testing, etc.
This is just a wrapper around all other Engines to keep them at the same place. This doesnt do anything except storing other Engines.
+This can be find in main.zig, in the main function.
The FileEngine is responsible for managing files, including reading and writing.
Most methods will parse all files of a struct and evaluate them with a filter and do stuff if true. For example parseEntities will parse all entities and if the filter return true,
+will write using the writer argument a JSON object with the entity's data.
Those methods are usually sperated into 2 methods. The main one and a OneFile version, e.g. parseEntitiesOneFile. The main one will call a thread for each file using multiple OneFile version.
+This is how multi-threading is done.
The SchemaEngine manage everything related to schemas.
This is mostly used to store a list of SchemaStruct, with is just one struct as defined in the schema. With all member names, data types, links, etc.
This is also here that I store the UUIDFileIndex, that is a map of UUID to file index. So I can quickly check if a UUID exist and in which file it is store.
+This work well but use a bit too much memory for me, around 220MB for 1_000_000 entities. I tried doing a Radix Trie but it doesn't use that much less memory, maybe I did a mistake somewhere.
The ThreadEngine manage the thread pool of the database.
This is also where is stored the ThreadSyncContext that is use for each OneFile version of parsing methods in the FileEngine. This is the only atomix value currently used in the database.
ZipponDB uses multi-threading to improve performance. Each struct is saved in multiple .zid files, and a thread pool is used to process files concurrently. Each thread has its own buffered writer, and the results are concatenated and sent once all threads finish.
The only shared atomic values between threads are the number of found structs and the number of finished threads. This approach keeps things simple and easy to implement, avoiding parallel threads accessing the same file.
+AdditionalData keep what is between []. It is composed of 2 struct AdditionalData and AdditionalDataMember.
AdditionalDataMember have the name of the member, it's position in the schema file and an AdditionalData.
AdditionalData have a limit (the first part [100]), and a list of AdditionalDataMember.
A filter is a series of condition. It use a tree approach, by that I mean the filter have a root node that is either a condition or have 2 others nodes (left and right).
+For example the filter {name = 'Bob'} have one root node that is the condition. So when I evaluate the struct, I just check this condition.
Now for like {name = 'Bob' AND age > 0}, the root node have as left node the condition name = 'Bob' and right node the condition age > 0.
To look like that: +
+A condition is part of filters. It is one 'unit of condition'. For example name = 'Bob' is one condition.
+name = 'Bob' and age > 0 are 2 conditions and one filter. It is created inside parseCondition in the ziqlParser.
A condition have those infos:
+32equal or inint or strDataIteratorNewData is a map with member name as key and ConditionValue as value, it is created when parsing and is use to add data into a file. +I transform ConditionValue into Zid Data. Maybe I can directly do a map member name -> zid Data ?
+A RelationMap is use when I need to return relationship. Let's say we have this query GRAB User [orders [date]].
The RelationMap have a struct_name, here Order. A member name, here orders. A map with UUID as key and a string as value.
When I first init the map, I am parsing the first struct (here User). So it populate the map with empty string for entities that I want to return. +Here it will be UUID of Order.
+Then I parse Order file and add the string to the right UUID, skipping UUID that are not in the map.
+Once that done, I parse the JSON response that I generated when parsing User. Where a relationship should be, there is {<|[16]u8|>}, where [16]u8 is the UUID of the entity that shoud be here.
+So now I can just replace it by the right key in the map.
This is responsable to transform the raw Data into a JSON, Table or other output format to send send to end user. +Basically the last step before sending.
+ + + + + + + + + + + + + +ZipponData is a library developped in the context of ZipponDB.
+The library intent to create a simple way to store and parse data from a file in the most efficient and fast way possible.
+There is 6 data type available in ZipponData:
+| Type | +Zig type | +Bytes in file | +
|---|---|---|
| int | +i32 | +4 | +
| float | +f64 | +8 | +
| bool | +bool | +1 | +
| str | +[]u8 | +4 + len | +
| uuid | +[16]u8 | +16 | +
| unix | +u64 | +8 | +
Each type have its array equivalent.
+createFileDataDataWriterDataIteratordeleteFileHere an example of how to use it: +
const std = @import("std");
+
+pub fn main() !void {
+ const allocator = std.testing.allocator;
+
+ // 0. Make a temporary directory
+ try std.fs.cwd().makeDir("tmp");
+ const dir = try std.fs.cwd().openDir("tmp", .{});
+
+ // 1. Create a file
+ try createFile("test", dir);
+
+ // 2. Create some Data
+ const data = [_]Data{
+ Data.initInt(1),
+ Data.initFloat(3.14159),
+ Data.initInt(-5),
+ Data.initStr("Hello world"),
+ Data.initBool(true),
+ Data.initUnix(2021),
+ };
+
+ // 3. Create a DataWriter
+ var dwriter = try DataWriter.init("test", dir);
+ defer dwriter.deinit(); // This just close the file
+
+ // 4. Write some data
+ try dwriter.write(&data);
+ try dwriter.write(&data);
+ try dwriter.write(&data);
+ try dwriter.write(&data);
+ try dwriter.write(&data);
+ try dwriter.write(&data);
+ try dwriter.flush(); // Dont forget to flush !
+
+ // 5. Create a schema
+ // A schema is how the iterator will parse the file.
+ // If you are wrong here, it will return wrong/random data
+ // And most likely an error when iterating in the while loop
+ const schema = &[_]DType{
+ .Int,
+ .Float,
+ .Int,
+ .Str,
+ .Bool,
+ .Unix,
+ };
+
+ // 6. Create a DataIterator
+ var iter = try DataIterator.init(allocator, "test", dir, schema);
+ defer iter.deinit();
+
+ // 7. Iterate over data
+ while (try iter.next()) |row| {
+ std.debug.print("Row: {any}\n", .{ row });
+ }
+
+ // 8. Delete the file (Optional ofc)
+ try deleteFile("test", dir);
+ try std.fs.cwd().deleteDir("tmp");
+}
+Note: The dir can be null and it will use cwd.
+All data type have an array equivalent. To write an array, you need to first encode it using allocEncodArray before writing it.
+This use an allocator so you need to free what it return.
When read, an array is just the raw bytes. To get the data itself, you need to create an ArrayIterator. Here an example:
pub fn main() !void {
+ const allocator = std.testing.allocator;
+
+ // 0. Make a temporary directory
+ try std.fs.cwd().makeDir("array_tmp");
+ const dir = try std.fs.cwd().openDir("array_tmp", .{});
+
+ // 1. Create a file
+ try createFile("test", dir);
+
+ // 2. Create and encode some Data
+ const int_array = [4]i32{ 32, 11, 15, 99 };
+ const data = [_]Data{
+ Data.initIntArray(try allocEncodArray.Int(allocator, &int_array)), // Encode
+ };
+ defer allocator.free(data[0].IntArray); // DOnt forget to free it
+
+ // 3. Create a DataWriter
+ var dwriter = try DataWriter.init("test", dir);
+ defer dwriter.deinit();
+
+ // 4. Write some data
+ try dwriter.write(&data);
+ try dwriter.flush();
+
+ // 5. Create a schema
+ const schema = &[_]DType{
+ .IntArray,
+ };
+
+ // 6. Create a DataIterator
+ var iter = try DataIterator.init(allocator, "test", dir, schema);
+ defer iter.deinit();
+
+ // 7. Iterate over data
+ var i: usize = 0;
+ if (try iter.next()) |row| {
+
+ // 8. Iterate over array
+ var array_iter = ArrayIterator.init(&row[0]); // Sub array iterator
+ while (array_iter.next()) |d| {
+ try std.testing.expectEqual(int_array[i], d.Int);
+ i += 1;
+ }
+
+ }
+
+ try deleteFile("test", dir);
+ try std.fs.cwd().deleteDir("array_tmp");
+}
+Done on a AMD Ryzen 7 7800X3D with a Samsung SSD 980 PRO 2TB (up to 7,000/5,100MB/s for read/write speed) on one thread.
+Data use: +
const schema = &[_]DType{
+ .Int,
+ .Float,
+ .Int,
+ .Str,
+ .Bool,
+ .Unix,
+};
+
+const data = &[_]Data{
+ Data.initInt(1),
+ Data.initFloat(3.14159),
+ Data.initInt(-5),
+ Data.initStr("Hello world"),
+ Data.initBool(true),
+ Data.initUnix(2021),
+};
+Result:
+| Number of Entity | +Total Write Time (ms) | +Average Write Time / entity (μs) | +Total Read Time (ms) | +Average Read Time / entity (μs) | +File Size (kB) | +
|---|---|---|---|---|---|
| 1 | +0.01 | +13.63 | +0.025 | +25.0 | +0.04 | +
| 10 | +0.01 | +1.69 | +0.03 | +3.28 | +0.4 | +
| 100 | +0.04 | +0.49 | +0.07 | +0.67 | +4.0 | +
| 1_000 | +0.36 | +0.36 | +0.48 | +0.48 | +40 | +
| 10_000 | +3.42 | +0.34 | +4.67 | +0.47 | +400 | +
| 100_000 | +36.39 | +0.36 | +48.00 | +0.49 | +4_000 | +
| 1_000_000 | +361.41 | +0.36 | +481.00 | +0.48 | +40_000 | +
Note: You can check benchmark to see performance of the real database using multi-threading. Was able to parse 1_000_000 users in less than 100ms
+ + + + + + + + + + + + + +ZipponDB is developped by a single person, Adrien Bouvais.
+I started this project because I wanted to learn a lot of things:
+I wanted to create a "real" project, not just some TODO app. But something original with actually everything. +That's why I added automatic documentation update when push in main. Or automatic build and release when I push a new tag. And why I make it look like a proper open-source project.
+I didn't research that much how other db do stuff. I wanted to just do it, figure it out and make silly mistake. That how you learn. +I may take this knowledge and try to do more niche database, something maybe a bit more through upfront. +Because here I just wrote the simpliest query I could think of that can do easy relationship. +But let's be honest, that's kind of a mess. I more want to do like a vector db or tigerbeetle and there transaction database.
+Or just just simply a SQL db. I mean I don't think it would be that difficult based on what I already did.
+I came into this project with 0 real knowledge of how a database work and a small week-end of coding zig. I didn't read that much about how other people make databases. +My goal was more to learn by mistake, doing it myself. I spended a lot of time at the beguining planing how the database will work inside, before coding.
+For the how itself, I solved one small problem at the time. 'How to open a file ?', 'How to read and write bytes ?', 'How to do a tokenizer ?', etc... And litle by litle, +you go closer and closer to your objectif.
+I learned a lot of thing with this project. And even if I would do some part differently if I do it again, I think it is a fairly decent piece of software.
+ + + + + + + + + + + + + +