current/doc/WritePerf.txt

writing performance test:

copying the 58970 records of unesb (~ 500 byte each)
with simple full index (-idxall)
making one entry for every field (total of 585334 fields)

system:
$
cat /proc/cpuinfo 
...
model name      : Pentium III (Coppermine)
cpu MHz         : 801.832
cache size      : 256 KB

free
             total       used       free     shared    buffers     cached
Mem:        256252     150428     105824          0        640      70560
-/+ buffers/cache:      79228     177024
Swap:       265032       7668     257364
$

initial write to non existing db db/test/t,
creating a 30MB .mst and 18MB .oxi
$
time ./openisis -db db/unesb/unesb -append db/test/t -fmt mfn -idxall >/dev/null

real    0m20.304s
user    0m15.680s
sys     0m3.600s

./openisis -db db/test/t -fmt mfn | wc -l
58970
./openisis -db db/test/t -ifdump | wc -l
585334
$

that is about 2,900 rows + 29,000 index entries per second.
(without indexing, it writes nearly 15,000 rows a second).

successive append are substantially slower, as IO cache is exhausted.


*       the never reading system

On a 650 MHz system with *lots* of cache memory
(it actually *never* reads a block),
a test of five successive appends yields pretty much constants times
around 29 seconds per pass (~ 2,000 rows + 20,000 entries / sec),
although the .mst stacks up to 294850 records and the index then has
2926670 entries.

That same system does on that fat db
one million random reads in 10 seconds (~ rows 100,000 / sec)
and an ifchk (252465 keys in 50596 blks of 1K depth 4) in 5.18 secs,
accessing 48,738 per second.


*       dual CPU use

Using both CPUs of this box,
four parallel writers (copying unesb to four different targets)
take a total of 60 seconds, giving a combined throughput of
more than 3,900 rows + 39,000 index entries / sec.


*       comparision

While these numbers are all derived with single threaded debugging code,
they yet outperform the (more or less) "relational" databases
by one or even two orders of magnitude.

For example, MySQL tested 350768 inserts without key in in 619*0.62 seconds,
yielding 914 rows/sec as compared 15,000 with OpenIsis.
Insert with key (one, probably) did less than 35 rows per second,
compared to around 3,000 (with ten keys each).
Select (probably reading all rows) did 20,000 in 39*0.54 seconds,
that is 950 per second -- compared to 100,000 with OpenIsis.

I couldn't figure out what kind of system was used for these
>       http://www.mysql.com/information/benchmark-results/result-db2,informix,ms-sql,mysql,solid,sybase-relative.html  MySQL benchmarks,
however, it probably was not much less than our 800 MHz Pentium.
Most other databases are even slower, and that, by the way,
is also our experience when working with Oracle and the like.
1	dpavlin	237	writing performance test:
2
3			copying the 58970 records of unesb (~ 500 byte each)
4			with simple full index (-idxall)
5			making one entry for every field (total of 585334 fields)
6
7			system:
8			$
9			cat /proc/cpuinfo
10			...
11			model name : Pentium III (Coppermine)
12			cpu MHz : 801.832
13			cache size : 256 KB
14
15			free
16			total used free shared buffers cached
17			Mem: 256252 150428 105824 0 640 70560
18			-/+ buffers/cache: 79228 177024
19			Swap: 265032 7668 257364
20			$
21
22			initial write to non existing db db/test/t,
23			creating a 30MB .mst and 18MB .oxi
24			$
25			time ./openisis -db db/unesb/unesb -append db/test/t -fmt mfn -idxall >/dev/null
26
27			real 0m20.304s
28			user 0m15.680s
29			sys 0m3.600s
30
31			./openisis -db db/test/t -fmt mfn \| wc -l
32			58970
33			./openisis -db db/test/t -ifdump \| wc -l
34			585334
35			$
36
37			that is about 2,900 rows + 29,000 index entries per second.
38			(without indexing, it writes nearly 15,000 rows a second).
39
40			successive append are substantially slower, as IO cache is exhausted.
41
42
43			* the never reading system
44
45			On a 650 MHz system with lots of cache memory
46			(it actually never reads a block),
47			a test of five successive appends yields pretty much constants times
48			around 29 seconds per pass (~ 2,000 rows + 20,000 entries / sec),
49			although the .mst stacks up to 294850 records and the index then has
50			2926670 entries.
51
52			That same system does on that fat db
53			one million random reads in 10 seconds (~ rows 100,000 / sec)
54			and an ifchk (252465 keys in 50596 blks of 1K depth 4) in 5.18 secs,
55			accessing 48,738 per second.
56
57
58			* dual CPU use
59
60			Using both CPUs of this box,
61			four parallel writers (copying unesb to four different targets)
62			take a total of 60 seconds, giving a combined throughput of
63			more than 3,900 rows + 39,000 index entries / sec.
64
65
66			* comparision
67
68			While these numbers are all derived with single threaded debugging code,
69			they yet outperform the (more or less) "relational" databases
70			by one or even two orders of magnitude.
71
72			For example, MySQL tested 350768 inserts without key in in 619*0.62 seconds,
73			yielding 914 rows/sec as compared 15,000 with OpenIsis.
74			Insert with key (one, probably) did less than 35 rows per second,
75			compared to around 3,000 (with ten keys each).
76			Select (probably reading all rows) did 20,000 in 39*0.54 seconds,
77			that is 950 per second -- compared to 100,000 with OpenIsis.
78
79			I couldn't figure out what kind of system was used for these
80			> http://www.mysql.com/information/benchmark-results/result-db2,informix,ms-sql,mysql,solid,sybase-relative.html MySQL benchmarks,
81			however, it probably was not much less than our 800 MHz Pentium.
82			Most other databases are even slower, and that, by the way,
83			is also our experience when working with Oracle and the like.