adrien/ZipponDB
1
0
Fork 0
Code Actions Releases Activity

Added docs

Browse Source
This commit is contained in:
Adrien Bouvais 2024-11-25 22:43:03 +01:00
parent 08cae48cbc
commit 2338cb6549
12 changed files with 293 additions and 75 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
.github/workflows.ci.yml vendored Normal file
View File

@ -0,0 +1,28 @@
name: ci
on:
push:
branches:
- v0.1.5
permissions:
contents: write
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Configure Git Credentials
run: |
git config user.name github-actions[bot]
git config user.email 41898282+github-actions[bot]@users.noreply.github.com
- uses: actions/setup-python@v5
with:
python-version: 3.x
- run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
- uses: actions/cache@v4
with:
key: mkdocs-material-${{ env.cache_id }}
path: .cache
restore-keys: |
mkdocs-material-
- run: pip install mkdocs-material
- run: mkdocs gh-deploy --force

10
Dockerfile Normal file
View File

@ -0,0 +1,10 @@
FROM scratch
COPY zig-out/bin/zippon /
COPY example.zipponschema /
ENV ZIPPONDB_PATH=data
ENV ZIPPONDB_SCHEMA=example.zipponschema
CMD ["/zippon"]

4
docs/Benchmark.md
View File

@ -40,7 +40,7 @@ This take 6.24GB space on disk, seperated into xx files of xx MB
| 12 | 5.1 | 85 |
| 16 | 4.3 | 100 |
![alt text](https://github.com/MrBounty/ZipponDB/blob/v0.1.4/python/charts/time_usage_per_thread_50_000_000.png)
![Chart](https://github.com/MrBounty/ZipponDB/blob/v0.1.5/python/charts/time_usage_per_thread_50_000_000.png)
## 1 000 000
This take 127MB space on disk, sperated into 24 files of 5.2MB
@ -56,7 +56,7 @@ This take 127MB space on disk, sperated into 24 files of 5.2MB
| 12 | 136 |
| 16 | 116 |
![alt text](https://github.com/MrBounty/ZipponDB/blob/v0.1.4/python/charts/time_usage_per_thread_1_000_000.png)
![Chart](https://github.com/MrBounty/ZipponDB/blob/v0.1.4/python/charts/time_usage_per_thread_1_000_000.png)
## TODO

14
docs/Data type.md Normal file
View File

@ -0,0 +1,14 @@
# Data types
There is 8 data types:
- `int`: 32 bit integer
- `float`: 64 bit float. Need to have a dot, `1.` is a float `1` is an integer
- `bool`: Boolean, can be `true` or `false`
- `string`: Character array between `''`
- `UUID`: Id in the UUID format, used for relationship, ect. All struct have an id member
- `date`: A date in yyyy/mm/dd
- `time`: A time in hh:mm:ss.mmmm
- `datetime`: A date time in yyyy/mm/dd-hh:mm:ss:mmmm
All data types can be an array of those types using `[]` in front of it. So `[]int` is an array of integer.

22
docs/Date and time.md Normal file
View File

@ -0,0 +1,22 @@
# Date and time
ZipponDB use 3 different date and time data type. Those are use like any other tpye like `int` or `float`.
## Date
Data type `date` represent a single day. To write a date, you use this format: `yyyy/mm/dd`.
Like that: `2024/10/19`.
## Time
Data type `time` represent a time of the day. To write a time, you use this format: `hh:mm:ss.mmmmm`.
Like that: `12:45:00.0000`.
Millisecond and second are optional so this work too: `12:45:00` and `12:45`
## Datetime
Data type `datetime` mix of both, it use this format: `yyyy/mm/dd-hh:mm:ss.mmmmm`.
Like that: `2024/10/19-12:45:00.0000`.
Millisecond and second are optional so this work too: `2024/10/19-12:45:00` and `2024/10/19-12:45`

0
docs/Quickstart.md Normal file
View File

67
docs/Roadmap.md Normal file
View File

@ -0,0 +1,67 @@
# Roadmap
***Note: This will probably evolve over time.***
### Alpha
#### v0.1 - Base
- [X] UUID
- [X] CLI
- [X] Tokenizers
- [X] ZiQL parser
- [X] Schema engine
- [X] File engine
#### v0.2 - Usable
- [ ] Relationships
- [ ] Arrays
- [X] Custom data file
- [X] Date
- [X] Logs
- [X] Query multi threading
#### v0.3 - QoL
- [ ] Schema migration
- [ ] Dump/Bump data
- [ ] Recovery
- [ ] Better CLI
- [ ] Linked query
### Beta
#### v0.4 - Usability
- [ ] Server
- [ ] Docker
- [ ] Config file
- [ ] Python interface
- [ ] Go interface
#### v0.5 - In memory
- [ ] In memory option
- [ ] Cache
#### v0.6 - Performance
- [ ] Transaction
- [ ] Other multi threading
- [ ] Query optimization
- [ ] Index
#### v0.7 - Safety
- [ ] Auth
- [ ] Metrics
- [ ] Durability
### Gold
#### v0.8 - Advanced
- [ ] Query optimizer
#### v0.9 - Docs
- [ ] ZiQL tuto
- [ ] Deployment tuto
- [ ] Code docs
- [ ] CLI help
#### v1.0 - Web interface
- [ ] Query builder
- [ ] Tables
- [ ] Schema visualization
- [ ] Dashboard metrics

55
docs/Schema.md Normal file
View File

@ -0,0 +1,55 @@
# Schema
In ZipponDB, you use structures, or structs for short, and not tables to organize how your data is stored and manipulated. A struct has a name like `User` and members like `name` and `age`.
## Create a Schema
ZipponDB use a seperate file to declare all structs to use in the database.
Here an example of a file:
```
User (
name: str,
email: str,
best_friend: User,
)
```
Note that `best_friend` is a link to another `User`.
Here is a more advanced example with multiple structs:
```
User (
name: str,
email: str,
friends: []User,
posts: []Post,
comments: []Comment,
)
Post (
title: str,
image: str,
at: date,
like_by: []User,
comments: []Comment,
)
Comment (
content: str,
at: date,
like_by: []User,
)
```
***Note: `[]` before the type means an array of this type.***
## Migration to a new schema - Not yet implemented
In the future, you will be able to update the schema, such as adding a new member to a struct, and update the database. For the moment, you can't change the schema once it's initialized.
## Commands
`schema init path/to/schema.file`: Init the database using a schema file.
`schema describe`: Print the schema use by the currently selected database.

144
docs/ZiQL.md
View File

@ -1,18 +1,18 @@
# ZipponQL
ZipponDB uses its own query language, ZipponQL or ZiQL for short. Here are the key points to remember:
- 4 actions available: `GRAB`, `ADD`, `UPDATE`, `DELETE`
- All queries start with an action followed by a struct name
- `{}` are filters
- `[]` specify how much and what data
- `()` contain new or updated data (not already in the file)
- `()` contain new or updated data (not already in files)
- `||` are additional options
- By default, all members that are not links are returned
- To return links or only some members, specify them between []
***Disclaimer: A lot of features are still missing, and the language may change over time.***
# Making errors
## Making errors
When you make an error writing ZiQL, you should see something like this to help you understand where you made a mistake:
```
@ -27,51 +27,69 @@ GRAB User {name = 'Bob' AND {age > 10}}
^
```
# Data types
## Filters
There is 5 data types for the moment:
- `int`: 64 bit integer
- `float`: 64 bit float. Need to have a dot, `1.` is a float `1` is an integer.
- `bool`: Boolean, can be `true` or `false`
- `string`: Character array between `''`
- `UUID`: Id in the UUID format, used for relationship, ect. All struct have an id member.
- `date`: A date in yyyy/mm/dd
- `time`: A time in hh:mm:ss.mmmm
- `datetime`: A date time in yyyy/mm/dd-hh:mm:ss:mmmm
What is between `{}` are filters. You can see it as a list of condition. This filter is use when parsing files and evaluate every struct one by one and return `true`
or `false`.
All data types can be an array of those types using `[]` in front of it. So `[]int` is an array of integer.
For example `{ name = 'Bob' }` will return `true` if the member `name` of the evaluated struct is equal to `Bob`. This is the most important thing in ZipponDB.
You can see it as `WHERE` in SQL.
All data types can also be `null`. Except arrays that can only be empty.
## Link query - Not yet implemented
# Examples
You can also link query. Each query returns a list of UUID of a specific struct. You can use it in the next query.
Here an example where I create a new `Comment` that I then append to the list of comment of one specific `User`.
```js
ADD Comment (content='Hello world', at=NOW, like_by=[]) => added_comment => UPDATE User {id = '000'} TO (comments APPEND added_comment)
```
## GRAB
The name between `=>` is the variable name of the list of UUID used for the next queries, you can have multiple one if the link has more than 2 queries.
You can also just use one `=>` but the list of UUID is discarded in that case.
This can be use with GRAB too. So you can create variable before making the query. Here an example:
```js
GRAB User {name = 'Bob'} => bobs =>
GRAB User {age > 18} => adults =>
GRAB User {IN adults AND !IN bobs}
```
Which is the same as:
```js
GRAB User {name != 'Bob' AND age > 18}
```
## Examples
### GRAB
The main action is `GRAB`, this will parse files and return data.
#### Basic
Here's how to return all `User` entities without any filtering:
```js
GRAB User
```
To get all `User` entities above 18 years old:
To get all `User` entities above 30 years old:
```js
GRAB User {age > 18}
GRAB User {age > 30}
```
To return only the `name` member of `User` entities:
```js
GRAB User [name] {age > 18}
GRAB User [name] {age > 30}
```
To return the 10 first `User` entities:
```js
GRAB User [10] {age > 18}
GRAB User [10] {age > 30}
```
You can combine these options:
```js
GRAB User [10; name] {age > 18}
GRAB User [10; name] {age > 30}
```
Use multiple conditions:
@ -84,7 +102,8 @@ GRAB queries return a list of JSON objects with the data inside, e.g:
[{id:"1e170a80-84c9-429a-be25-ab4657894653", name: "Gwendolyn Ray", age: 70, email: "austin92@example.org", scores: [ 77 ], friends: [], }, ]
```
#### Not yet implemented
#### Ordering
**Not yet implemented**
To order the results by `name`:
```js
@ -96,71 +115,82 @@ You can specify how much data to return and which members to include, even for l
GRAB User [10; friends [1; name]]
```
##### Using IN
#### Array
You can use the `IN` operator to check if something is in an array:
```js
GRAB User { age > 10 AND name IN ['Adrien' 'Bob']}
```
This also works by using other filters. Here I get `User` entities that have a best friend named Adrien:
TODO: More
#### Relationship
2 main things to remember with relationship:
- You can use the dot `.` to refer to a relationship.
- You can use filter inside filter.
Get `User` that have a best friend named Adrien:
```js
GRAB User { bestfriend IN { name = 'Adrien' } }
```
---
When using an array with `IN`, it will return all User entities that have AT LEAST one friend named Adrien:
When using `IN`, it return all `User` that have AT LEAST one friend named Adrien:
```js
GRAB User { friends IN { name = 'Adrien' } }
```
---
To get `User` entities with all friends named Adrien:
```js
GRAB User { friends ALLIN { name = 'Adrien' } }
```
---
You can use `IN` on itself. Here I get all `User` entities that liked a `Comment` from 2024. Both queries return the same result:
```js
GRAB User { IN Comment {at > '2024/01/01'}.like_by}
GRAB Comment.like_by { at > '2024/01/01'}
```
You can obtain a similar result with this query, but it will return a list of `Comment` entities with a `liked_by` member that is similar to the `User` entities above. If you take all `liked_by` members inside all `Comment` entities, it will be the same list, but you can end up with duplicates since one `User` can like multiple `Comment` entities.
```js
GRAB Comment [liked_by] {at > '2024/01/01'}
```
##### Return relationship
---
You can also return a relationship only. The filter will be applied to `User` entities, but will return `Comment` entities:
```js
GRAB User.comments {name = 'Bob'}
```
---
You can do it as much as you like. This will return all `User` that liked comments from Bob:
```js
GRAB User.comments.like_by {name = 'Bob'}
```
---
This can also be used inside filters. Note that we need to specify `User` because it is a different struct than `Post`. Here, I get all `Post` entities that have a comment from Bob:
```js
GRAB Post {comments IN User{name = 'Bob'}.comments}
```
---
You can also do the same but only for the first Bob found:
```js
GRAB Post {comments IN User [1] {name = 'Bob'}.comments}
```
---
Be careful; this will return all `User` entities that liked a comment from 10 `User` entities named Bob:
Be careful; this will return all `User` that liked a comment from 10 `User` named Bob:
```js
GRAB User.comments.like_by [10] {name = 'Bob'}
```
---
To get 10 `User` entities that liked a comment from any `User` entity named Bob, you need to use:
To get 10 `User` that liked a comment from any `User` named Bob, you need to use:
```js
GRAB User.comments.like_by [comments [like_by [10]]] {name = 'Bob'}
```
##### Using !
#### !
You can use `!` to return the opposite. When used with `IN`, it checks if something is NOT in the list. When used with filters, it returns entities that do not match the filter.
This will return all `User` entities that didn't like a `Comment` in 2024:
@ -178,7 +208,7 @@ Which is the same as:
GRAB Comment.like_by { at < '2024/01/01'}
```
## ADD
### ADD
The `ADD` action adds one entity to the database. The syntax is similar to `GRAB`, but uses `()`. This signifies that the data is not yet in the database.
@ -187,7 +217,7 @@ Here's an example:
ADD User (name = 'Bob', age = 30, email = 'bob@email.com', scores = [1 100 44 82])
```
You need to specify all members when adding an entity (default values are coming soon).
You need to specify all members when adding an entity (default values in roadmap).
The `ADD` query will return a list of added IDs, e.g.:
@ -195,7 +225,7 @@ The `ADD` query will return a list of added IDs, e.g.:
["1e170a80-84c9-429a-be25-ab4657894653", "1e170a80-84c9-429a-be25-ab4657894654", ]
```
#### Not yet implemented
**Not yet implemented**
And you can also add them in batch
```js
@ -207,7 +237,7 @@ You don't need to specify the members in the second entity as long as the order
ADD User (name = 'Bob', age = 30, email = 'bob@email.com', scores = [1 100 44 82]) ('Bob2', 33, 'bob2@email.com', [])
```
## DELETE
### DELETE
Similar to `GRAB` but deletes all entities found using the filter and returns a list of deleted UUIDs.
```js
@ -219,7 +249,7 @@ The `DELETE` query will return a list of deleted IDs, e.g.:
["1e170a80-84c9-429a-be25-ab4657894653", "1e170a80-84c9-429a-be25-ab4657894654", ]
```
## UPDATE
### UPDATE
A mix of `GRAB` and `ADD`. It takes a filter first, then the new data.
Here, we update the first 5 `User` entities named 'adrien' to capitalize the name and become 'Adrien':
@ -234,7 +264,7 @@ The `UPDATE` query will return a list of updated IDs, e.g.:
["1e170a80-84c9-429a-be25-ab4657894653", "1e170a80-84c9-429a-be25-ab4657894654", ]
```
#### Not yet implemented
**Not yet implemented**
You can use operations on values themselves when updating:
```js
@ -264,34 +294,4 @@ UPDATE User {name='Bob'} TO (comments REMOVE { at < '2023/12/31'})
I may include more options later.
# Date
***WIP***
## Date
To write a date, you use this format: `yyyy/mm/dd`.
Like that: `2024/10/19`.
***Note: You cant use negative years***
## Time
To write a time, you use this format: `hh:mm:ss.mmmmm`.
Like that: `12:45:00.0000`.
Millisecond and second are optional so this work too: `12:45:00` and `12:45`
## Datetime
Mix of both, to write a datetime, you use this format: `yyyy/mm/dd-hh:mm:ss.mmmmm`.
Like that: `2024/10/19-12:45:00.0000`.
Millisecond and second are optional so this work too: `2024/10/19-12:45:00` and `2024/10/19-12:45`
GRAB User {birthday > 2020/10/19 AND birthday < 2024/01/01 }
GRAB IOT {id = '0000-0000' AND .TemperatureSensor.Temperature.timestamp > 22-45-50.0000}
GRAB IOT {.TemperatureSensor.TemperatureRecord IN TemperatureRecord{.timestamp > 22-45-50.0000 AND version = 3}}
GRAB IOT {.TemperatureSensor.TemperatureRecord IN .{timestamp > 22-45-50.0000} AND .TemperatureSensor IN .{version = 3}}

11
docs/index.md Normal file
View File

@ -0,0 +1,11 @@
# ZipponDB Intro
ZipponDB is a relational database written entirely in Zig from scratch with 0 dependencies.
ZipponDB's goal is to be ACID, light, simple, and high-performance. It aims at small to medium applications that don't need fancy features but a simple and reliable database.
### Why Zippon ?
- Relational database (Soon)
- Simple and minimal query language
- Small, light, fast, and implementable everywhere

3
example.zipponschema
View File

@ -6,5 +6,6 @@ User (
last_order: datetime,
a_time: time,
scores: []int,
friends: []str,
best_friend: User,
friends: []User,
)

10
mkdocs.yml Normal file
View File

@ -0,0 +1,10 @@
site_name: ZipponDB Documentation
site_url: https://docs.bouvai.com
theme:
name: readthedocs
markdown_extensions:
- attr_list
- md_in_html
- pymdownx.blocks.caption
plugins:
- glightbox