Chris's Wiki :: blog/tech/SSDBenchmarkingConcerns Commentshttps://utcc.utoronto.ca/~cks/space/blog/tech/SSDBenchmarkingConcerns?atomcommentsDWiki2014-08-09T16:41:32ZRecent comments in Chris's Wiki :: blog/tech/SSDBenchmarkingConcerns.By GlacJAY on /blog/tech/SSDBenchmarkingConcernstag:CSpace:blog/tech/SSDBenchmarkingConcerns:5f680f2276f69b0752c5a947e480593122b301cbGlacJAYhttp://glacjay.info<div class="wikitext"><p>You can try to write the sequence of 7's times, such as 7, 14, 21, 28, ..., as 4 or 8 bytes. The point is not the randomness, but uncompressable.</p>
</div>2014-08-09T16:41:32ZBy Zev Weiss on /blog/tech/SSDBenchmarkingConcernstag:CSpace:blog/tech/SSDBenchmarkingConcerns:e3b5d0a3055adc50c537fcb370d7ee7539c111d1Zev Weiss<div class="wikitext"><p>And as a small follow-up to my previous comment, I see <code>aes-128-cbc</code> wasn't even a particularly good choice -- <code>aes-128-xts</code> and <code>aes-128-ctr</code> can each do more like ~1.8GB/s on the same hardware.</p>
</div>2014-08-02T19:04:22ZBy Josef "Jeff" Sipek on /blog/tech/SSDBenchmarkingConcernstag:CSpace:blog/tech/SSDBenchmarkingConcerns:e236d9dfe3ef9329a897f650cdfec2cc01069da1Josef "Jeff" Sipekhttp://blahg.josefsipek.net<div class="wikitext"><p>I was going to comment, but the comment turned out to be large enough that instead I wrote up a blahg post of my own: <a href="http://blahg.josefsipek.net/?p=500">http://blahg.josefsipek.net/?p=500</a></p>
</div>2014-08-02T17:06:12ZBy Christian Neukirchen on /blog/tech/SSDBenchmarkingConcernstag:CSpace:blog/tech/SSDBenchmarkingConcerns:b5776737e6cb8dba26605498be77eb3b92ca65f1Christian Neukirchen<div class="wikitext"><p>As a high-speed source for random data, you can use my tool "rdd": <a href="https://github.com/chneukirchen/rdd">https://github.com/chneukirchen/rdd</a> (does 677MB/s on a Core i5-2400 here)</p>
</div>2014-08-02T14:39:28ZBy Ewen McNeill on /blog/tech/SSDBenchmarkingConcernstag:CSpace:blog/tech/SSDBenchmarkingConcerns:d5422506202716ece03a683ebae770e232609753Ewen McNeill<div class="wikitext"><p>Possible work around (for now): generate a block of random data that is <em>just</em> longer than the internal block of the device (eg SSD erase block), say by a few bytes -- 128kB + 2 bytes. The write that out repeatedly, as if the data block was wrap-around at the end. The result is that each block begins in a slightly different position in the "random" stream, so trivial block by block deduplication is not possible, and each block itself is "random", so trivial in-block compression is not possible either. Compared with /dev/null there is more work in shuffling partial blocks around, but it's not that much slower, especially with a scatter/gather write interface. And you only pay the "random" overhead once up front.</p>
<p>If that still doesn't seem random enough, you could XOR each block with say the block number as you go (but that will require CPU, not just scatter/gather so could be CPU limited).</p>
<p>Both of these have the advantage that given the initial 128kB + 2 bytes, and algorithm, the rest is predictable so you could validate what is read back too.</p>
<p>Ewen</p>
</div>2014-08-02T09:56:38ZBy gigaboo@gmail.com on /blog/tech/SSDBenchmarkingConcernstag:CSpace:blog/tech/SSDBenchmarkingConcerns:2b060d4f1bd47000cab8c9c5798da1a023c48a14gigaboo@gmail.com<div class="wikitext"><p>I conduct my I/O with a large (>5GB) video file read from RAM. This provides a non-compressible data set that is reliably repeatable without using slow pseudo-random tools.</p>
<p>In restricted memory situations, I use smaller segments of the file that fit into the working memory set.</p>
</div>2014-08-02T09:50:46ZBy Zev Weiss on /blog/tech/SSDBenchmarkingConcernstag:CSpace:blog/tech/SSDBenchmarkingConcerns:0dae6953a2a3d01f085f81f5694226f541e03099Zev Weiss<div class="wikitext"><p>I've often run into the same problem (generating "random-enough" data at high speed). What I usually end up doing is just taking <code>/dev/zero</code> and running it through one of the faster ciphers offered by <code>openssl enc</code> (make sure not to use an ECB mode though!). My new Haswell box can generate ~670MB/s or so this way using <code>aes-128-cbc</code>, for example.</p>
<p>(Relatedly, one thing I've had on my TODO list for some time is to write a simple command-line [P]RNG offering a range of selectable algorithms/sources at varying points along the quality-vs-speed spectrum.)</p>
</div>2014-08-02T09:09:02Z