nodejs JavaScript堆内存不足使用实践测试RNG

Question

我写了一个小的种子RNG，现在我正在尝试使用pracrand对它进行一些统计测试。rng在这里：https://github.com/nomadcrypto/seededrsa/blob/master/rng.js，测试代码是：

const RNG = require("./rng");

function test() {
    const seed = "praise you muffin lion enable neck grocery crumble super myself license ghost"
    const rng = new RNG(seed)

    while(true) {
        var bytes = Buffer.allocUnsafe(8)
        rng.nextBytes(bytes)
        process.stdout.write(bytes.toString("binary"))

    }


}

test()

我像这样运行测试: node /path/to/ test _rng.js | /path/to/RNG_test stdin8

这应该是将8位随机二进制字符串输出到stdout，并且pracrand正在从stdin读取它们。这将始终在同一点终止，并显示以下错误

RNG_test using PractRand version 0.93
RNG = RNG_stdin8, seed = 0x61ce6e4b
test set = normal, folding = standard (8 bit)

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

Writing Node.js report to file: report.20190311.183904.23092.001.json
Node.js report completed
 1: 0x959570 node::Abort() [node]
 2: 0x95a25e node::OnFatalError(char const*, char const*) [node]
 3: 0xb3735e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xb37594 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xf36bd2  [node]
 6: 0xf36cd8 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [node]
 7: 0xf433f8 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
 8: 0xf43f0b v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 9: 0xf46c41 v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [node]
10: 0xf10e74 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [node]
11: 0x11c72be v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [node]
12: 0x82556f4fc5d 
rng=RNG_stdin8, seed=0x61ce6e4b
length= 1 megabyte (2^20 bytes), time= 34.0 seconds
  Test Name                         Raw       Processed     Evaluation
  BCFN(2+0,13-7,T)                  R=+39522  p = 0           FAIL !!!!!!!!  
  BCFN(2+1,13-7,T)                  R=+40616  p = 0           FAIL !!!!!!!!  
  BCFN(2+2,13-8,T)                  R=+43491  p = 0           FAIL !!!!!!!!  
  BCFN(2+3,13-8,T)                  R=+32793  p =  2e-8323    FAIL !!!!!!!!  
  BCFN(2+4,13-8,T)                  R=+20180  p =  1e-5122    FAIL !!!!!!!!  
  BCFN(2+5,13-9,T)                  R=+12114  p =  4e-2723    FAIL !!!!!!!!  
  BCFN(2+6,13-9,T)                  R= +5993  p =  7e-1348    FAIL !!!!!!!!  
  DC6-9x1Bytes-1                    R= +6430  p =  8e-3715    FAIL !!!!!!!!  
  Gap-16:A                          R=+19027  p = 0           FAIL !!!!!!!!  
  Gap-16:B                          R=+80036  p = 0           FAIL !!!!!!!!  
  FPF-14+6/16:(0,14-4)              R=+23224  p = 0           FAIL !!!!!!!!  
  FPF-14+6/16:(1,14-5)              R=+26476  p = 0           FAIL !!!!!!!!  
  FPF-14+6/16:(2,14-5)              R= +3407  p =  2e-2824    FAIL !!!!!!!!  
  FPF-14+6/16:(3,14-6)              R= +2417  p =  6e-1850    FAIL !!!!!!!!  
  FPF-14+6/16:(4,14-7)              R= +1720  p =  1e-1368    FAIL !!!!!!!!  
  FPF-14+6/16:(5,14-8)              R= +1161  p =  2.8e-835   FAIL !!!!!!!   
  FPF-14+6/16:(6,14-8)              R=+527.7  p =  9.0e-380   FAIL !!!!!!!   
  FPF-14+6/16:(7,14-9)              R=+758.3  p =  9.9e-478   FAIL !!!!!!!   
  FPF-14+6/16:(8,14-10)             R=+545.9  p =  5.5e-291   FAIL !!!!!!    
  FPF-14+6/16:(9,14-11)             R=+670.1  p =  6.4e-293   FAIL !!!!!!    
  FPF-14+6/16:all                   R=+29837  p = 0           FAIL !!!!!!!!  
  FPF-14+6/16:all2                  R=+306094057 p = 0           FAIL !!!!!!!!  
  FPF-14+6/16:cross                 R= +6279  p =  2e-4805    FAIL !!!!!!!!
  ...and 22 test result(s) without anomalies

我敢肯定这是相当沉重的。有没有人能提供一些提示或技巧来帮助我正确测试这个rng？

Answer 1

解决方案是使用流api并只推送缓冲区，而不是打印二进制字符串

const RNG = require("./rng");

var crypto= require("crypto");
var stream = require('stream');
var util = require('util');

var Readable = stream.Readable;

function RandomStream(options) {
  if (!(this instanceof RandomStream)) {
    return new RandomStream(options);
  }
  Readable.call(this, options);

  this.seed = "praise you muffin lion enable neck grocery crumble super myself license ghost"
  this.rng = new RNG(this.seed)
  this.bytes = Buffer.allocUnsafe(8);

}
util.inherits(RandomStream, Readable);

RandomStream.prototype._read = function (size) {
  var ready = true;
  while (ready) { 
    var bytes = this.rng.randomBytes(8)
    //bytes = crypto.randomBytes(8)
    ready = this.push(bytes)
  }

};

var readstream = new RandomStream();
readstream.pipe(process.stdout)

这与我的rng的randomBytes和crypto.randomBytes都能正常工作。到目前为止，在实践中没有发现任何东西

(env) nomadcrypto@nomadcrypto:~/projects/seededrsa$ node test_rng.js |/home/nomadcrypto/projects/practrand/build/RNG_test stdin8
RNG_test using PractRand version 0.93
RNG = RNG_stdin8, seed = 0x3bb19275
test set = normal, folding = standard (8 bit)

rng=RNG_stdin8, seed=0x3bb19275
length= 4 megabytes (2^22 bytes), time= 2.3 seconds
  no anomalies in 56 test result(s)

rng=RNG_stdin8, seed=0x3bb19275
length= 8 megabytes (2^23 bytes), time= 5.0 seconds
  no anomalies in 60 test result(s)

rng=RNG_stdin8, seed=0x3bb19275
length= 16 megabytes (2^24 bytes), time= 9.9 seconds
  no anomalies in 66 test result(s)

rng=RNG_stdin8, seed=0x3bb19275
length= 32 megabytes (2^25 bytes), time= 19.1 seconds
  no anomalies in 72 test result(s)

rng=RNG_stdin8, seed=0x3bb19275
length= 64 megabytes (2^26 bytes), time= 38.2 seconds
  no anomalies in 76 test result(s)

rng=RNG_stdin8, seed=0x3bb19275
length= 128 megabytes (2^27 bytes), time= 76.0 seconds
  no anomalies in 82 test result(s)

rng=RNG_stdin8, seed=0x3bb19275
length= 256 megabytes (2^28 bytes), time= 147 seconds
  no anomalies in 88 test result(s)

rng=RNG_stdin8, seed=0x3bb19275
length= 512 megabytes (2^29 bytes), time= 289 seconds
  no anomalies in 92 test result(s)

rng=RNG_stdin8, seed=0x3bb19275
length= 1 gigabyte (2^30 bytes), time= 571 seconds
  no anomalies in 98 test result(s)