diff --git a/README.md b/README.md index 6cd26b6..7cc26a2 100644 --- a/README.md +++ b/README.md @@ -2,26 +2,28 @@ # Introduction -ZipponDB is a relational database written entirely in Zig from stractch with 0 dependency. +ZipponDB is a relational database written entirely in Zig from scratch with 0 dependency. -ZipponDB goal is to be ACID, light, simple and high performance. It aim small to medium application that don't need fancy features but a simple and reliable database. +ZipponDB goal is to be ACID, light, simple and high performance. It aim small to medium application that don't need fancy features but a simple and reliable database. ### Why Zippon ? -- Open-source and written 100% in Zig with 0 dependency -- Relational database +- Relational database (Soon) - Simple and minimal query language - Small, light, fast and implementable everywhere # Quickstart -You can build the binary directly from the source code (tuto is comming), or using the binary in the release (comming too). +1. **Get a binary:** You can build the binary directly from the source code for any architecture (tuto is comming), or using the binary in the release (comming too). +2. **Create a database:** You can then run the binary, this will start a Command Line Interface. The first thing to do is to create a new database. For that run the command `db new path/to/directory`, +it will create a `ZipponDB` directory. Then `database metrics` to see if it worked. +3. **Select a database:** You can select a database by using `db use path/to/ZipponDB`. You can also set the environment variable ZIPPONDB_PATH and it will and use this path, +this need to be the path to a directory with proper DATA, BACKUP and LOG directory. +4. **Attach a schema:** Once the database created, you need to attach a schema to it (see next section for how to define a schema). For that you can run `schema init path/to/schema.txt`. +This will create new directories and empty files used to store data. You can test the current db schema by running `schema describe`. +5. **Use the database:** You can now start using the database by sending query like that: `run "ADD User (name = 'Bob')"`. -You can then run it, starting a Command Line Interface. The first thing to do is to create a new database. For that run the command `db new path/to/folder`, it will create a `ZipponDB` folder with multiple stuffs inside. Then `database metrics` to see if it worked. You can change between database by using `db swap path/to/ZipponDB`. - -Once the database created, you need to attach a schema to it (see next section for how to define a schema). For that you can run `schema init path/to/schema.txt`. This will create new folder and empty files used to store data. - -You can now start using the database by sending query like that: `run "ADD User (name = 'Bob')"`. +***Note: For the moment ZipponDB use the current working directory as main directory so all path are a sub_path of it.*** # Declare a schema @@ -67,9 +69,7 @@ Comment ( Note: `[]` before the type mean a list/array of this type. -### Migration to a new schema - -***Not yet implemented*** +### Migration to a new schema - Not yet implemented In the future, you will be able to update the schema like add a new member to a struct and update the database. For the moment, you can't change the schema once init. @@ -79,212 +79,61 @@ ZipponDB use it's own query language, ZipponQL or ZiQL for short. Here the keys - 4 actions available: `GRAB` `ADD` `UPDATE` `DELETE` - All query start with an action then a struct name -- `{}` Are filters -- `[]` Are how much; what data -- `()` Are new or updated data (Not already in file) -- `||` Are additional options -- By default all member that are not link are return -- To return link or only some members, specify them between `[]` +- `{}` For filters +- `[]` For how much; what data +- `()` For new or updated data (Not already in file) +- `||` For additional options ***Disclaimer: Lot of stuff are still missing and the language may change over time.*** -## GRAB +## Quickstart -The main action is `GRAB`, this will parse files and return data. +**For more information see [ZiQL Introduction](https://github.com/MrBounty/ZipponDB/blob/main/ZiQL.md)** -Here how to return all `User` without any filtering: -```js -GRAB User -``` +### GRAB -To get all `User` above 18 years old: -```js -GRAB User {age > 18} -``` - -To only return the name of `User`: -```js -GRAB User [name] {age > 18} -``` - -To return the 10 first `User`: -```js -GRAB User [10] {age > 18} -``` - -You can use both: -```js -GRAB User [10; name] {age > 18} -``` - -To order it using the name: -```js -GRAB User [10; name] {age > 10} |ASC name| -``` - -Use multiple condition: +The main action is `GRAB`, this will parse files and return data. ```js GRAB User {name = 'Bob' AND (age > 30 OR age < 10)} ``` -#### Not yet implemented - -You can specify how much and what to return even for link inside struct. In this example I get 1 friend name for 10 `User`: -```js -GRAB User [10; friends [1; name]] +GRAB query return a list of JSON with the data inside, e.g: +``` +[{id:"1e170a80-84c9-429a-be25-ab4657894653", name: "Gwendolyn Ray", age: 70, email: "austin92@example.org", scores: [ 77 ], friends: [], }, ] ``` -##### Using IN -You can use the `IN` operator to check if something is in an array: -```js -GRAB User { age > 10 AND name IN ['Adrien' 'Bob']} -``` +### ADD -This also work by using other filter. Here I get `User` that have a best friend named Adrien: -```js -GRAB User { bestfriend IN { name = 'Adrien' } } -``` - -When using an array with IN, it will return all `User` that have at least ONE friend named Adrien: -```js -GRAB User { friends IN { name = 'Adrien' } } -``` - -To get `User` with ALL friends named Adrien: -```js -GRAB User { friends ALLIN { name = 'Adrien' } } -``` - -You can use `IN` on itself. Here I get all `User` that liked a `Comment` that is from 2024. Both queries return the same thing: -```js -GRAB User { IN Comment {at > '2024/01/01'}.like_by} -GRAB Comment.like_by { at > '2024/01/01'} -``` - -You can optain a similar result with this query but it will return a list of `Comment` with a member `liked_by` that is similar to `User` above. If you take all `liked_by` inside all `Comment`, it will be the same list but you can end up with duplicate as one `User` can like multiple `Comment`. -```js -GRAB Comment [liked_by] {at > '2024/01/01'} -``` - -##### Return relationship - -You can also return a relationship only. The filter will be done on `User` but will return `Comment`: -```js -GRAB User.comments {name = 'Bob'} -``` - -You can do it as much as you like. This will return all `User` that liked comments from Bob: -```js -GRAB User.comments.like_by {name = 'Bob'} -``` - -This can also be use inside filter. Note that we need to specify `User` because it is a different struct that `Post`. Here I get all `Post` that have a comment from Bob: -```js -GRAB Post {comments IN User{name = 'Bob'}.comments} -``` - -Can also do the same but only for the first Bob found: -```js -GRAB Post {comments IN User [1] {name = 'Bob'}.comments} -``` - -Be carefull, this will return all `User` that liked a comment from 10 `User` named Bob: -```js -GRAB User.comments.like_by [10] {name = 'Bob'} -``` - -To get 10 `User` that liked a comment from any `User` named Bob, you need to use: -```js -GRAB User.comments.like_by [comments [like_by [10]]] {name = 'Bob'} -``` - -##### Using ! -You can use `!` to return the opposite. Use with `IN`, it check if it is NOT is the list. Use it with filters, it return entities that do not respect the filter. - -This will return all `User` that didn't like a `Comment` in 2024: -```js -GRAB User { !IN Comment {at > '2024/01/01'}.like_by} -``` - -Be carefull because this do not return the same, it return all `User` that liked a `Comment` not in 2024: -```js -GRAB Comment.like_by !{ at > '2024/01/01'} -``` - -Which is the same as: -```js -GRAB Comment.like_by { at < '2024/01/01'} -``` - -## ADD - -The `ADD` action will add one entity into the database. -The synthax is similare but use `()`, this mean that the data is not yet in the database. - -Here an example: +The `ADD` action will add one entity into the database. ```js ADD User (name = 'Bob', age = 30, email = 'bob@email.com', scores = [1 100 44 82]) ``` -You need to specify all member when adding an entity (default value are comming). +### DELETE -#### Not yet implemented - -And you can also add them in batch -```js -ADD User (name = 'Bob', age = 30, email = 'bob@email.com', scores = [1 100 44 82]) (name = 'Bob2', age = 33, email = 'bob2@email.com', scores = []) -``` - -You don't need to specify the member in the second entity as long as the order is respected. -```js -ADD User (name = 'Bob', age = 30, email = 'bob@email.com', scores = [1 100 44 82]) ('Bob2', 33, 'bob2@email.com', []) -``` - -## DELETE - -Similare to `GRAB` but delete all entity found using the filter and return the list of UUID deleted. +Similare to `GRAB` but delete all entity found using the filter. ```js DELETE User {name = 'Bob'} ``` -## UPDATE +### UPDATE A mix of `GRAB` and `ADD`. This take a filter first, then the new data. -Here we update the 5 first User named `adrien` to add a capital and become `Adrien`. +Here we update the 5 first User named `bob` to add a capital and become `Bob`. ```js -UPDATE User [5] {name='adrien'} TO (name = 'Adrien') +UPDATE User [5] {name='bob'} TO (name = 'Bob') ``` -Note that compared to `ADD`, you don't need to specify all member between `()`. Only the one specify will be updated. +### Not yet implemented -#### Not yet implemented +A lot of things are not yet implemented, you can find examples in the [ZiQL Introduction](https://github.com/MrBounty/ZipponDB/blob/main/ZiQL.md). -You can use operations on itself too when updating: -```js -UPDATE User {name = 'Bob'} TO (age += 1) -``` - -You can also manipulate array, like adding or removing values. -```js -UPDATE User {name='Bob'} TO (scores APPEND 45) -UPDATE User {name='Bob'} TO (scores REMOVEAT [0 1 2]) -``` - -For now there is 4 keywords to manipulate list: -- `APPEND`: Add value at the end of the list. -- `REMOVE`: Check the list and if the same value is found, delete it. -- `REMOVEAT`: Delete the value at a specific index. -- `CLEAR`: Remove all value in the array. - -Except `CLEAR` that take no value, each can use one value or an array of value, if chose an array it will perform the operation on all value in the array. - -For relationship, you can use filter on it: -```js -UPDATE User {name='Bob'} TO (comments APPEND {id = '000'}) -UPDATE User {name='Bob'} TO (comments REMOVE { at < '2023/12/31'}) -``` - -I may include more options later. +This include: +- Relationship +- Ordering +- Batch +- Array manipulation +- And more... ## Link query - Not yet implemented @@ -314,11 +163,15 @@ All data type can be an array of those type using [] in front of it. So []int is All data type can also be `null`. Expect array that can only be empty. -# Lexique +# Why I created it ? -- **Struct:** A struct of how to store data. E.g. `User` -- **Entity:** An entity is one instance of a struct. -- **Member:** A member is one data saved in a struct. E.g. `name` in `User` +Well the first reason is to learn, both zig and databases. + +The second is to use it in my apps. I like to deploy Golang + HTMX app on Fly.io but I often find myself struggelling to get a simple database. +I can either host it myself but I need to link my app and the db securely. Or use a cloud db service but that mean my db is far from my app. +All I want is to give to a Fly machine 10go of storage, do some backup on it and call it a day. But for that I need to include it to the Dockerfile of my app, what easier way than just a binary ? + +So that my goal long term, to use it in my apps as a simple database that live WITH the app, sharing CPU and memory. # How does it work ? @@ -326,19 +179,21 @@ TODO: Create a tech doc of what is happening inside. # Roadmap +***Note: This will probably evolve over time.*** + #### v0.1 - Base - [X] UUID - [X] CLI - [X] Tokenizers -- [ ] ZiQL parser -- [ ] Schema engine +- [X] ZiQL parser +- [X] Schema engine - [X] File engine #### v0.2 - Usable - [ ] B+Tree - [ ] Relationships - [ ] Date -- [ ] Link query +- [ ] Linked query - [ ] Docker #### v0.3 - QoL @@ -346,9 +201,11 @@ TODO: Create a tech doc of what is happening inside. - [ ] Dump/Bump data - [ ] Recovery - [ ] Better CLI +- [ ] Logs #### v0.4 - Usability - [ ] Server +- [ ] Config file - [ ] Python interface - [ ] Go interface @@ -360,6 +217,7 @@ TODO: Create a tech doc of what is happening inside. - [ ] Transaction - [ ] Multi threading - [ ] Lock manager +- [ ] Optimized data file #### v0.7 - Safety - [ ] Auth diff --git a/ZiQL.md b/ZiQL.md new file mode 100644 index 0000000..2b6796d --- /dev/null +++ b/ZiQL.md @@ -0,0 +1,252 @@ +# ZipponQL + +ZipponDB use it's own query language, ZipponQL or ZiQL for short. Here the keys point to remember: + +- 4 actions available: `GRAB` `ADD` `UPDATE` `DELETE` +- All query start with an action then a struct name +- `{}` Are filters +- `[]` Are how much; what data +- `()` Are new or updated data (Not already in file) +- `||` Are additional options +- By default all member that are not link are return +- To return link or only some members, specify them between `[]` + +***Disclaimer: Lot of stuff are still missing and the language may change over time.*** + +# Making erros + +When you do an error writting ZiQL, you should see something like this to help you understand where you did a mistake: +``` +Error: Expected string +GRAB User {name = Bob} + ^^^ +``` + +``` +Error: Expected ( or member name. +GRAB User {name = 'Bob' AND {age > 10}} + ^ +``` + +# Examples + +## GRAB + +The main action is `GRAB`, this will parse files and return data. + +Here how to return all `User` without any filtering: +```js +GRAB User +``` + +To get all `User` above 18 years old: +```js +GRAB User {age > 18} +``` + +To only return the name of `User`: +```js +GRAB User [name] {age > 18} +``` + +To return the 10 first `User`: +```js +GRAB User [10] {age > 18} +``` + +You can use both: +```js +GRAB User [10; name] {age > 18} +``` + +Use multiple condition: +```js +GRAB User {name = 'Bob' AND (age > 30 OR age < 10)} +``` + +GRAB query return a list of JSON with the data inside, e.g: +``` +[{id:"1e170a80-84c9-429a-be25-ab4657894653", name: "Gwendolyn Ray", age: 70, email: "austin92@example.org", scores: [ 77 ], friends: [], }, ] +``` + +#### Not yet implemented + +To order it using the name: +```js +GRAB User [10; name] {age > 10} |ASC name| +``` + +You can specify how much and what to return even for link inside struct. In this example I get 1 friend name for 10 `User`: +```js +GRAB User [10; friends [1; name]] +``` + +##### Using IN +You can use the `IN` operator to check if something is in an array: +```js +GRAB User { age > 10 AND name IN ['Adrien' 'Bob']} +``` + +This also work by using other filter. Here I get `User` that have a best friend named Adrien: +```js +GRAB User { bestfriend IN { name = 'Adrien' } } +``` + +When using an array with IN, it will return all `User` that have at least ONE friend named Adrien: +```js +GRAB User { friends IN { name = 'Adrien' } } +``` + +To get `User` with ALL friends named Adrien: +```js +GRAB User { friends ALLIN { name = 'Adrien' } } +``` + +You can use `IN` on itself. Here I get all `User` that liked a `Comment` that is from 2024. Both queries return the same thing: +```js +GRAB User { IN Comment {at > '2024/01/01'}.like_by} +GRAB Comment.like_by { at > '2024/01/01'} +``` + +You can optain a similar result with this query but it will return a list of `Comment` with a member `liked_by` that is similar to `User` above. If you take all `liked_by` inside all `Comment`, it will be the same list but you can end up with duplicate as one `User` can like multiple `Comment`. +```js +GRAB Comment [liked_by] {at > '2024/01/01'} +``` + +##### Return relationship + +You can also return a relationship only. The filter will be done on `User` but will return `Comment`: +```js +GRAB User.comments {name = 'Bob'} +``` + +You can do it as much as you like. This will return all `User` that liked comments from Bob: +```js +GRAB User.comments.like_by {name = 'Bob'} +``` + +This can also be use inside filter. Note that we need to specify `User` because it is a different struct that `Post`. Here I get all `Post` that have a comment from Bob: +```js +GRAB Post {comments IN User{name = 'Bob'}.comments} +``` + +Can also do the same but only for the first Bob found: +```js +GRAB Post {comments IN User [1] {name = 'Bob'}.comments} +``` + +Be carefull, this will return all `User` that liked a comment from 10 `User` named Bob: +```js +GRAB User.comments.like_by [10] {name = 'Bob'} +``` + +To get 10 `User` that liked a comment from any `User` named Bob, you need to use: +```js +GRAB User.comments.like_by [comments [like_by [10]]] {name = 'Bob'} +``` + +##### Using ! +You can use `!` to return the opposite. Use with `IN`, it check if it is NOT is the list. Use it with filters, it return entities that do not respect the filter. + +This will return all `User` that didn't like a `Comment` in 2024: +```js +GRAB User { !IN Comment {at > '2024/01/01'}.like_by} +``` + +Be carefull because this do not return the same, it return all `User` that liked a `Comment` not in 2024: +```js +GRAB Comment.like_by !{ at > '2024/01/01'} +``` + +Which is the same as: +```js +GRAB Comment.like_by { at < '2024/01/01'} +``` + +## ADD + +The `ADD` action will add one entity into the database. +The synthax is similare but use `()`, this mean that the data is not yet in the database. + +Here an example: +```js +ADD User (name = 'Bob', age = 30, email = 'bob@email.com', scores = [1 100 44 82]) +``` + +You need to specify all member when adding an entity (default value are comming). + +#### Not yet implemented + +ADD query return a list ids added, e.g: +``` +["1e170a80-84c9-429a-be25-ab4657894653", "1e170a80-84c9-429a-be25-ab4657894654", ] +``` + +And you can also add them in batch +```js +ADD User (name = 'Bob', age = 30, email = 'bob@email.com', scores = [1 100 44 82]) (name = 'Bob2', age = 33, email = 'bob2@email.com', scores = []) +``` + +You don't need to specify the member in the second entity as long as the order is respected. +```js +ADD User (name = 'Bob', age = 30, email = 'bob@email.com', scores = [1 100 44 82]) ('Bob2', 33, 'bob2@email.com', []) +``` + +## DELETE + +Similare to `GRAB` but delete all entity found using the filter and return the list of UUID deleted. +```js +DELETE User {name = 'Bob'} +``` + +#### Not yet implemented + +DELETE query return a list ids deleted, e.g: +``` +["1e170a80-84c9-429a-be25-ab4657894653", "1e170a80-84c9-429a-be25-ab4657894654", ] +``` + +## UPDATE + +A mix of `GRAB` and `ADD`. This take a filter first, then the new data. +Here we update the 5 first User named `adrien` to add a capital and become `Adrien`. +```js +UPDATE User [5] {name='adrien'} TO (name = 'Adrien') +``` + +Note that compared to `ADD`, you don't need to specify all member between `()`. Only the one specify will be updated. + +#### Not yet implemented + +UPDATE query return a list ids updated, e.g: +``` +["1e170a80-84c9-429a-be25-ab4657894653", "1e170a80-84c9-429a-be25-ab4657894654", ] +``` + +You can use operations on itself too when updating: +```js +UPDATE User {name = 'Bob'} TO (age += 1) +``` + +You can also manipulate array, like adding or removing values. +```js +UPDATE User {name='Bob'} TO (scores APPEND 45) +UPDATE User {name='Bob'} TO (scores APPEND [45 99]) +UPDATE User {name='Bob'} TO (scores REMOVEAT [0 1 2]) +``` + +For now there is 4 keywords to manipulate list: +- `APPEND`: Add value at the end of the list. +- `REMOVE`: Check the list and if the same value is found, delete it. +- `REMOVEAT`: Delete the value at a specific index. +- `CLEAR`: Remove all value in the array. + +Except `CLEAR` that take no value, each can use one value or an array of value, if chose an array it will perform the operation on all value in the array. + +For relationship, you can use filter on it: +```js +UPDATE User {name='Bob'} TO (comments APPEND {id = '000'}) +UPDATE User {name='Bob'} TO (comments REMOVE { at < '2023/12/31'}) +``` + +I may include more options later. diff --git a/build.zig b/build.zig index 1c5a337..2223fd2 100644 --- a/build.zig +++ b/build.zig @@ -3,7 +3,7 @@ const std = @import("std"); pub fn build(b: *std.Build) void { // Build part const target = b.standardTargetOptions(.{}); - const optimize = b.standardOptimizeOption(.{}); + const optimize = b.standardOptimizeOption(.{ .preferred_optimize_mode = .ReleaseFast }); const exe = b.addExecutable(.{ .name = "zippon", .root_source_file = b.path("src/cli.zig"), diff --git a/example.zipponschema b/example.zipponschema index 14ce3e6..8782eb5 100644 --- a/example.zipponschema +++ b/example.zipponschema @@ -3,5 +3,5 @@ User ( age: int, email: str, scores: []int, - friends: []bool, + friends: []str, )