From 30a7a83e564391adc24236fb1f4c2320d4274e60 Mon Sep 17 00:00:00 2001 From: MrBounty Date: Thu, 30 Jan 2025 21:21:02 +0100 Subject: [PATCH] Update some docs --- docs/Benchmark.md | 604 +++++++++++++++++++++++++++++++++++++++------- docs/index.md | 9 +- 2 files changed, 523 insertions(+), 90 deletions(-) diff --git a/docs/Benchmark.md b/docs/Benchmark.md index 2f125b3..effaa38 100644 --- a/docs/Benchmark.md +++ b/docs/Benchmark.md @@ -1,121 +1,557 @@ # Benchmark -***Benchmark are set to quicly evolve. I have currently multiple ideas to improve performance.*** +***Benchmark are set to evolve. I have currently multiple ideas to improve performance.*** + +ZipponDB is fairly fast and can easely query millions of entities. +Current limitation is around 5GB I would say, depending of the use and kind of data saved. +After that query can start to become slow as multiple optimization are still missing. + +Most of query's time is writing entities into a JSON string. Parsing the file itself take little time. +For example in the benchmark report bellow, I parse 100 000 users in around 40ms if there is no entities to send and 130ms if all 100 000 entities are to send. + +I choosed to release ZipponDB binary with the small release. Zig has a fast and safe release, but the fast release isn't that much faster, if not at all. ## Command -You can run `zig build benchmark` if you clone the repo to benchmark your machine. [More info.](/ZipponDB/build) +You can run `zig build benchmark`, if you clone the repo to benchmark your machine. [More info.](/ZipponDB/build) -Here an example on my machine: +Here an example on my machine with 16 core: ``` ===================================== -Populating with 500000 users. -Populate duration: 8.605314 seconds +Populating with 5000 users. +Populate duration: 0.035698 seconds -Database path: benchmark -Total size: 50.99Mb -LOG: 0.00Mb +Database path: benchmarkDB +Total size: 0.36Mb +CPU core: 16 +Max file size: 5.00Mb +LOG: 0.02Mb BACKUP: 0.00Mb -DATA: 50.93Mb - User: 50.92683124542236Mb 500000 entities +DATA: 0.33Mb + Item: 0.00Mb | 19 entities | 1 files + User: 0.33Mb | 5000 entities | 1 files + Order: 0.00Mb | 0 entities | 1 files + Category: 0.00Mb | 4 entities | 1 files -------------------------------------- -Query: GRAB User {} -Duration: 457.758686 ms +Query: GRAB User {} +Time: 16.90 ± 25.22 ms | Min 8.25ms | Max 92.55ms -Query: GRAB User [1] {} -Duration: 1.285849 ms +Query: GRAB User {name='asd'} +Time: 2.62 ± 0.10 ms | Min 2.52ms | Max 2.85ms -Query: GRAB User [name] {} -Duration: 138.041888 ms +Query: GRAB User [1] {} +Time: 0.16 ± 0.01 ms | Min 0.15ms | Max 0.18ms -Query: GRAB User {name = 'Charlie'} -Duration: 63.094060 ms +Query: GRAB User [name] {} +Time: 7.88 ± 11.69 ms | Min 3.91ms | Max 42.94ms -Query: GRAB User {age > 30} -Duration: 335.654647 ms +Query: GRAB User {name = 'Charlie'} +Time: 3.87 ± 0.16 ms | Min 3.70ms | Max 4.17ms -Query: GRAB User {bday > 2000/01/01} -Duration: 52.896498 ms +Query: GRAB Category {} +Time: 0.20 ± 0.07 ms | Min 0.17ms | Max 0.41ms -Query: GRAB User {age > 30 AND name = 'Charlie' AND bday > 2000/01/01} -Duration: 56.295173 ms +Query: GRAB Item {} +Time: 0.21 ± 0.02 ms | Min 0.19ms | Max 0.25ms -Query: GRAB User {best_friend IN {name = 'Charlie'}} -Duration: 69.165272 ms +Query: GRAB Order {} +Time: 0.14 ± 0.01 ms | Min 0.13ms | Max 0.18ms -Query: DELETE User {} -Duration: 93.530622 ms +Query: GRAB Order [from, items, quantity, at] {} +Time: 4.18 ± 12.01 ms | Min 0.15ms | Max 40.21ms +Query: DELETE User {} +Time: 0.64 ± 1.13 ms | Min 0.23ms | Max 4.04ms + +Read: 1907698 Entity/second *Include small condition +Write: 350200 Entity/second + +===================================== + +Populating with 100000 users. +Populate duration: 0.707605 seconds + +Database path: benchmarkDB +Total size: 6.62Mb +CPU core: 16 +Max file size: 5.00Mb +LOG: 0.02Mb +BACKUP: 0.00Mb +DATA: 6.59Mb + Item: 0.00Mb | 19 entities | 1 files + User: 6.59Mb | 100000 entities | 2 files + Order: 0.00Mb | 0 entities | 1 files + Category: 0.00Mb | 4 entities | 1 files + +-------------------------------------- + +Query: GRAB User {} +Time: 126.99 ± 3.05 ms | Min 123.37ms | Max 133.56ms + +Query: GRAB User {name='asd'} +Time: 38.12 ± 1.60 ms | Min 36.48ms | Max 41.88ms + +Query: GRAB User [1] {} +Time: 0.19 ± 0.02 ms | Min 0.16ms | Max 0.22ms + +Query: GRAB User [name] {} +Time: 59.33 ± 1.29 ms | Min 58.02ms | Max 61.47ms + +Query: GRAB User {name = 'Charlie'} +Time: 53.29 ± 1.00 ms | Min 51.50ms | Max 54.78ms + +Query: GRAB Category {} +Time: 0.19 ± 0.01 ms | Min 0.18ms | Max 0.22ms + +Query: GRAB Item {} +Time: 5.51 ± 13.43 ms | Min 0.22ms | Max 45.22ms + +Query: GRAB Order {} +Time: 0.16 ± 0.01 ms | Min 0.15ms | Max 0.18ms + +Query: GRAB Order [from, items, quantity, at] {} +Time: 0.17 ± 0.02 ms | Min 0.15ms | Max 0.21ms + +Query: DELETE User {} +Time: 5.96 ± 17.04 ms | Min 0.26ms | Max 57.07ms + +Read: 2623338 Entity/second *Include small condition +Write: 1125278 Entity/second + +===================================== + +Populating with 1000000 users. +Populate duration: 7.029142 seconds + +Database path: benchmarkDB +Total size: 65.96Mb +CPU core: 16 +Max file size: 5.00Mb +LOG: 0.02Mb +BACKUP: 0.00Mb +DATA: 65.93Mb + Item: 0.00Mb | 19 entities | 1 files + User: 65.93Mb | 1000000 entities | 14 files + Order: 0.00Mb | 0 entities | 1 files + Category: 0.00Mb | 4 entities | 1 files + +-------------------------------------- + +Query: GRAB User {} +Time: 250.77 ± 6.74 ms | Min 247.08ms | Max 270.61ms + +Query: GRAB User {name='asd'} +Time: 67.90 ± 0.42 ms | Min 67.31ms | Max 68.78ms + +Query: GRAB User [1] {} +Time: 8.92 ± 24.86 ms | Min 0.55ms | Max 83.51ms + +Query: GRAB User [name] {} +Time: 110.08 ± 5.27 ms | Min 106.86ms | Max 125.21ms + +Query: GRAB User {name = 'Charlie'} +Time: 73.65 ± 2.79 ms | Min 69.24ms | Max 79.22ms + +Query: GRAB Category {} +Time: 0.19 ± 0.04 ms | Min 0.16ms | Max 0.33ms + +Query: GRAB Item {} +Time: 0.21 ± 0.02 ms | Min 0.19ms | Max 0.26ms + +Query: GRAB Order {} +Time: 0.15 ± 0.01 ms | Min 0.14ms | Max 0.17ms + +Query: GRAB Order [from, items, quantity, at] {} +Time: 0.17 ± 0.01 ms | Min 0.16ms | Max 0.18ms + +Query: DELETE User {} +Time: 11.74 ± 34.19 ms | Min 0.29ms | Max 114.30ms + +Read: 14727354 Entity/second *Include small condition +Write: 5468517 Entity/second + +===================================== + +Populating with 10000000 users. +Populate duration: 72.675680 seconds + +Database path: benchmarkDB +Total size: 659.33Mb +CPU core: 16 +Max file size: 5.00Mb +LOG: 0.02Mb +BACKUP: 0.00Mb +DATA: 659.30Mb + Item: 0.00Mb | 19 entities | 1 files + User: 659.30Mb | 10000000 entities | 132 files + Order: 0.00Mb | 0 entities | 1 files + Category: 0.00Mb | 4 entities | 1 files + +-------------------------------------- + +Query: GRAB User {} +Time: 2535.29 ± 86.92 ms | Min 2448.39ms | Max 2712.78ms + +Query: GRAB User {name='asd'} +Time: 684.75 ± 39.96 ms | Min 649.09ms | Max 797.13ms + +Query: GRAB User [1] {} +Time: 6.65 ± 1.00 ms | Min 5.36ms | Max 8.75ms + +Query: GRAB User [name] {} +Time: 1106.21 ± 33.57 ms | Min 1056.57ms | Max 1172.61ms + +Query: GRAB User {name = 'Charlie'} +Time: 690.56 ± 20.41 ms | Min 661.51ms | Max 718.07ms + +Query: GRAB Category {} +Time: 0.21 ± 0.03 ms | Min 0.18ms | Max 0.31ms + +Query: GRAB Item {} +Time: 0.23 ± 0.04 ms | Min 0.19ms | Max 0.32ms + +Query: GRAB Order {} +Time: 0.15 ± 0.01 ms | Min 0.13ms | Max 0.17ms + +Query: GRAB Order [from, items, quantity, at] {} +Time: 0.17 ± 0.02 ms | Min 0.15ms | Max 0.21ms + +Query: DELETE User {} +Time: 109.55 ± 326.64ms | Min 0.47ms | Max 1089.46ms + +Read: 14603810 Entity/second *Include small condition +Write: 5403847 Entity/second ===================================== ``` -## Cloud +## Other core count -I started to benchmark using fly.io +Here some benchmark with different number of cpu core. -Using a 1x shared cpu I get around 40ms to parse and write 5000 users, not bad. +Note that don't show 5000 users because it use a single file so there is no multi threading as it is done one thread per file. -## File Parsing +### 1 -The majority of time taken by query come more from writting the response rather than parsing files. +``` +===================================== -In this example I create a random dataset of Users using this schema: -```lua -User ( - name: str, - age: int, - email: str, - bday: date, - last_order: datetime, - a_time: time, - scores: []int, - friends: []str, -) +Populating with 100000 users. +Populate duration: 0.697690 seconds + +Database path: benchmarkDB +Total size: 6.62Mb +CPU core: 1 +Max file size: 5.00Mb +LOG: 0.02Mb +BACKUP: 0.00Mb +DATA: 6.59Mb + Item: 0.00Mb | 19 entities | 1 files + User: 6.59Mb | 100000 entities | 2 files + Order: 0.00Mb | 0 entities | 1 files + Category: 0.00Mb | 4 entities | 1 files + +-------------------------------------- + +Query: GRAB User {} +Time: 169.31 ± 5.39 ms | Min 163.25ms | Max 181.23ms + +Query: GRAB User {name='asd'} +Time: 49.66 ± 1.14 ms | Min 48.23ms | Max 51.71ms + +Query: GRAB User [1] {} +Time: 0.20 ± 0.02 ms | Min 0.18ms | Max 0.25ms + +Query: GRAB User [name] {} +Time: 75.81 ± 1.01 ms | Min 74.15ms | Max 77.42ms + +Query: GRAB User {name = 'Charlie'} +Time: 73.54 ± 1.00 ms | Min 72.04ms | Max 75.61ms + +Query: GRAB Category {} +Time: 0.16 ± 0.01 ms | Min 0.15ms | Max 0.19ms + +Query: GRAB Item {} +Time: 6.37 ± 18.54 ms | Min 0.18ms | Max 62.01ms + +Query: GRAB Order {} +Time: 0.15 ± 0.01 ms | Min 0.13ms | Max 0.18ms + +Query: GRAB Order [from, items, quantity, at] {} +Time: 0.16 ± 0.02 ms | Min 0.14ms | Max 0.19ms + +Query: DELETE User {} +Time: 8.07 ± 23.43 ms | Min 0.22ms | Max 78.35ms + +Read: 2013655 Entity/second *Include small condition +Write: 835770 Entity/second + +===================================== + +Populating with 1000000 users. +Populate duration: 7.031715 seconds + +Database path: benchmarkDB +Total size: 65.96Mb +CPU core: 1 +Max file size: 5.00Mb +LOG: 0.02Mb +BACKUP: 0.00Mb +DATA: 65.93Mb + Item: 0.00Mb | 19 entities | 1 files + User: 65.93Mb | 1000000 entities | 14 files + Order: 0.00Mb | 0 entities | 1 files + Category: 0.00Mb | 4 entities | 1 files + +-------------------------------------- + +Query: GRAB User {} +Time: 1720.31 ± 31.89 ms | Min 1674.61ms | Max 1787.15ms + +Query: GRAB User {name='asd'} +Time: 511.09 ± 5.51 ms | Min 501.73ms | Max 521.17ms + +Query: GRAB User [1] {} +Time: 4.34 ± 10.89 ms | Min 0.66ms | Max 37.02ms + +Query: GRAB User [name] {} +Time: 788.89 ± 6.62 ms | Min 779.85ms | Max 802.79ms + +Query: GRAB User {name = 'Charlie'} +Time: 763.96 ± 5.05 ms | Min 752.04ms | Max 771.09ms + +Query: GRAB Category {} +Time: 0.19 ± 0.04 ms | Min 0.15ms | Max 0.28ms + +Query: GRAB Item {} +Time: 0.21 ± 0.02 ms | Min 0.19ms | Max 0.24ms + +Query: GRAB Order {} +Time: 0.13 ± 0.01 ms | Min 0.12ms | Max 0.14ms + +Query: GRAB Order [from, items, quantity, at] {} +Time: 0.16 ± 0.01 ms | Min 0.15ms | Max 0.19ms + +Query: DELETE User {} +Time: 78.98 ± 236.07ms | Min 0.24ms | Max 787.18ms + +Read: 1956584 Entity/second *Include small condition +Write: 826982 Entity/second + +===================================== + +Populating with 10000000 users. +Populate duration: 74.074046 seconds + +Database path: benchmarkDB +Total size: 659.33Mb +CPU core: 1 +Max file size: 5.00Mb +LOG: 0.02Mb +BACKUP: 0.00Mb +DATA: 659.30Mb + Item: 0.00Mb | 19 entities | 1 files + User: 659.30Mb | 10000000 entities | 132 files + Order: 0.00Mb | 0 entities | 1 files + Category: 0.00Mb | 4 entities | 1 files + +-------------------------------------- + +Query: GRAB User {} +Time: 18117.45 ± 106.80ms | Min 17946.04ms | Max 18349.54ms + +Query: GRAB User {name='asd'} +Time: 5141.26 ± 67.78 ms | Min 5059.89ms | Max 5275.02ms + +Query: GRAB User [1] {} +Time: 5.71 ± 0.22 ms | Min 5.34ms | Max 6.27ms + +Query: GRAB User [name] {} +Time: 7863.72 ± 26.90 ms | Min 7823.04ms | Max 7911.72ms + +Query: GRAB User {name = 'Charlie'} +Time: 7724.32 ± 38.54 ms | Min 7671.13ms | Max 7812.13ms + +Query: GRAB Category {} +Time: 0.21 ± 0.07 ms | Min 0.15ms | Max 0.38ms + +Query: GRAB Item {} +Time: 0.22 ± 0.07 ms | Min 0.18ms | Max 0.42ms + +Query: GRAB Order {} +Time: 0.14 ± 0.01 ms | Min 0.13ms | Max 0.17ms + +Query: GRAB Order [from, items, quantity, at] {} +Time: 0.17 ± 0.03 ms | Min 0.14ms | Max 0.25ms + +Query: DELETE User {} +Time: 796.37 ± 2387.51ms | Min 0.41ms | Max 7958.91ms + +Read: 1945047 Entity/second *Include small condition +Write: 770643 Entity/second +===================================== ``` -Here a user example: -```lua -run "ADD User (name = 'Diana Lopez',age = 2,email = 'allisonwilliams@example.org',scores=[37 85 90 71 88 85 68],friends = [],bday=1973/11/13,last_order=1979/07/18-15:05:26.590261,a_time=03:04:06.862213) +### 4 + ``` +===================================== -Let's do a query that parse all file but dont return anything, so we have the time to read and evaluate file but not writting and sending output. -```lua -run "GRAB User {name = 'asdfqwer'}" +Populating with 100000 users. +Populate duration: 0.700273 seconds + +Database path: benchmarkDB +Total size: 6.62Mb +CPU core: 4 +Max file size: 5.00Mb +LOG: 0.02Mb +BACKUP: 0.00Mb +DATA: 6.59Mb + Item: 0.00Mb | 19 entities | 1 files + User: 6.59Mb | 100000 entities | 2 files + Order: 0.00Mb | 0 entities | 1 files + Category: 0.00Mb | 4 entities | 1 files + +-------------------------------------- + +Query: GRAB User {} +Time: 131.92 ± 1.79 ms | Min 129.06ms | Max 135.24ms + +Query: GRAB User {name='asd'} +Time: 38.23 ± 1.12 ms | Min 36.52ms | Max 40.35ms + +Query: GRAB User [1] {} +Time: 0.17 ± 0.02 ms | Min 0.15ms | Max 0.22ms + +Query: GRAB User [name] {} +Time: 60.60 ± 1.52 ms | Min 58.62ms | Max 64.49ms + +Query: GRAB User {name = 'Charlie'} +Time: 53.90 ± 1.46 ms | Min 52.44ms | Max 56.96ms + +Query: GRAB Category {} +Time: 0.17 ± 0.02 ms | Min 0.15ms | Max 0.21ms + +Query: GRAB Item {} +Time: 3.70 ± 10.47 ms | Min 0.19ms | Max 35.10ms + +Query: GRAB Order {} +Time: 0.13 ± 0.00 ms | Min 0.13ms | Max 0.14ms + +Query: GRAB Order [from, items, quantity, at] {} +Time: 0.18 ± 0.07 ms | Min 0.15ms | Max 0.39ms + +Query: DELETE User {} +Time: 6.00 ± 17.36 ms | Min 0.21ms | Max 58.08ms + +Read: 2615841 Entity/second *Include small condition +Write: 1067359 Entity/second + +===================================== + +Populating with 1000000 users. +Populate duration: 6.912029 seconds + +Database path: benchmarkDB +Total size: 65.96Mb +CPU core: 4 +Max file size: 5.00Mb +LOG: 0.02Mb +BACKUP: 0.00Mb +DATA: 65.93Mb + Item: 0.00Mb | 19 entities | 1 files + User: 65.93Mb | 1000000 entities | 14 files + Order: 0.00Mb | 0 entities | 1 files + Category: 0.00Mb | 4 entities | 1 files + +-------------------------------------- + +Query: GRAB User {} +Time: 580.46 ± 7.08 ms | Min 566.39ms | Max 591.97ms + +Query: GRAB User {name='asd'} +Time: 166.33 ± 3.12 ms | Min 161.76ms | Max 171.96ms + +Query: GRAB User [1] {} +Time: 2.88 ± 6.81 ms | Min 0.53ms | Max 23.32ms + +Query: GRAB User [name] {} +Time: 262.20 ± 5.78 ms | Min 255.59ms | Max 273.91ms + +Query: GRAB User {name = 'Charlie'} +Time: 190.25 ± 5.76 ms | Min 184.78ms | Max 204.83ms + +Query: GRAB Category {} +Time: 0.18 ± 0.04 ms | Min 0.16ms | Max 0.29ms + +Query: GRAB Item {} +Time: 0.22 ± 0.02 ms | Min 0.19ms | Max 0.28ms + +Query: GRAB Order {} +Time: 0.14 ± 0.01 ms | Min 0.13ms | Max 0.17ms + +Query: GRAB Order [from, items, quantity, at] {} +Time: 0.16 ± 0.01 ms | Min 0.15ms | Max 0.19ms + +Query: DELETE User {} +Time: 26.91 ± 79.86 ms | Min 0.26ms | Max 266.51ms + +Read: 6012012 Entity/second *Include small condition +Write: 2414729 Entity/second + +===================================== + +Populating with 10000000 users. +Populate duration: 69.969231 seconds + +Database path: benchmarkDB +Total size: 659.33Mb +CPU core: 4 +Max file size: 5.00Mb +LOG: 0.02Mb +BACKUP: 0.00Mb +DATA: 659.30Mb + Item: 0.00Mb | 19 entities | 1 files + User: 659.30Mb | 10000000 entities | 132 files + Order: 0.00Mb | 0 entities | 1 files + Category: 0.00Mb | 4 entities | 1 files + +-------------------------------------- + +Query: GRAB User {} +Time: 5306.69 ± 34.07 ms | Min 5226.70ms | Max 5344.36ms + +Query: GRAB User {name='asd'} +Time: 1475.91 ± 24.85 ms | Min 1430.52ms | Max 1509.27ms + +Query: GRAB User [1] {} +Time: 4.73 ± 0.30 ms | Min 4.42ms | Max 5.32ms + +Query: GRAB User [name] {} +Time: 2374.83 ± 21.66 ms | Min 2343.64ms | Max 2410.61ms + +Query: GRAB User {name = 'Charlie'} +Time: 1556.61 ± 31.91 ms | Min 1512.58ms | Max 1622.71ms + +Query: GRAB Category {} +Time: 0.21 ± 0.06 ms | Min 0.16ms | Max 0.37ms + +Query: GRAB Item {} +Time: 0.20 ± 0.01 ms | Min 0.19ms | Max 0.24ms + +Query: GRAB Order {} +Time: 0.14 ± 0.01 ms | Min 0.13ms | Max 0.17ms + +Query: GRAB Order [from, items, quantity, at] {} +Time: 0.19 ± 0.05 ms | Min 0.14ms | Max 0.34ms + +Query: DELETE User {} +Time: 246.23 ± 737.02ms | Min 0.43ms | Max 2457.29ms + +Read: 6775468 Entity/second *Include small condition +Write: 2610439 Entity/second +===================================== ``` - -## 50 000 000 Users -This take 6.24GB space on disk. - -| Thread | Time (s) | Usage (%) | -| --- | --- | --- | -| 1 | 40 | 10 | -| 2 | 21 | 18 | -| 3 | 15 | 25 | -| 4 | 12 | 30 | -| 6 | 8.3 | 45 | -| 8 | 6.6 | 55 | -| 12 | 5.1 | 85 | -| 16 | 4.3 | 100 | - -![Chart](images/time_usage_per_thread_50_000_000.png) - -## 1 000 000 -This take 127MB space on disk, sperated into 24 files of 5.2MB - -| Thread | Time (ms) | -| --- | --- | -| 1 | 790 | -| 2 | 446 | -| 3 | 326 | -| 4 | 255 | -| 6 | 195 | -| 8 | 155 | -| 12 | 136 | -| 16 | 116 | - -![Chart](images/time_usage_per_thread_1_000_000.png) - diff --git a/docs/index.md b/docs/index.md index f917e52..70dba41 100644 --- a/docs/index.md +++ b/docs/index.md @@ -26,16 +26,13 @@ medium applications that want a quick and simple database. ## Key Features * **Small Binary:** ~300kb. -* **Fast:** Parse millions of entities in milliseconds. +* **Fast:** Parse millions of entities in milliseconds.* * **Relationship:** Build with focus on easy relationship. -* **Query Language:** Use it's own simple query language. +* **Query Language:** Use it's own stupid query language. * **No dependencies:** Depend on nothing, every line of code running is in the codebase and written for ZipponDB. * **Open-source:** Open-source under MIT licence. -* **Portable:** Easily compiled and deployed across various platforms.* -* **Low memory and safe:** Low memory footprint. (~8Mb / 100k entities)**. -* Plan for more platforms like arm, 32 bit system. -** Plan for optimizations. +* Check benchmark. ### Planned