加快将大量JSON数据写入磁盘

Question

我有一个循环，大概有40k个记录，基本上是呼叫。文件的写入似乎真的很慢。如果我在循环中迭代几乎是即时的，那么我就会意识到它不是核心数据迭代，而是文件编写过程缓慢。是否有比我在这里所做的更好的将数据流到文件的方法？

#ifndef jsonAppend
#define jsonAppend(X) [outStream write:[[X dataUsingEncoding:NSUTF8StringEncoding ] bytes] maxLength:[X lengthOfBytesUsingEncoding:NSUTF8StringEncoding]]
#endif



NSOutputStream *outStream = [NSOutputStream outputStreamToFileAtPath:tempFilePath append:NO];

    dispatch_async(backgroundQueue, ^{

// Open an output stream to write to.
[outStream open];

// Other code ... blablabla

for (AHRSMessage *msg in results)
{

    @autoreleasepool
    {

        [NSJSONSerialization writeJSONObject:msg.attributesAsDictionary toStream:outStream options:0 error:&error];
        jsonAppend(@",\n");

        i++;  // MessageCounter


        // Update UI only sometimes
        if (i % 100)
        {

            dispatch_async(dispatch_get_main_queue(), ^
            {
                @autoreleasepool {


                    float pct = i / recordCount ;
                    NSString *pctStr = [NSString stringWithFormat:@"%02.0f%%", pct * 100.0];
                    [[weakSelf percentLabel] setText:pctStr];
                    [[weakSelf parsingProgress] setProgress:pct animated:YES];

                    /* - Animate inner circle on units of 1% - */
                    int singPctMsgCount = recordCount / 100;
                    float fastParse = (i % singPctMsgCount) / (float)singPctMsgCount;
                    [[weakSelf fastParsingProgress] setProgress:fastParse animated:YES]     ;

                    [weakSelf updateUI];

                }
            });

        }

    }
} // end for loop

})；

Answer 1

因为它只有18‘s，所以只需将其序列化到一个NSMutableData对象，然后将其写入磁盘。

那应该是非常快的。NSMutableData可以处理大量甚至不适合闪存的数据(假设您有64位处理器)，更不用说iOS设备上的RAM了。

如下所示：

dispatch_async(backgroundQueue, ^{

  NSMutableData *outData = [NSMutableData data];

  // Other code ... blablabla

  for (AHRSMessage *msg in results)
  {

      @autoreleasepool
      {
          [outData appendData:[NSJSONSerialization dataWithJSONObject:msg.attributesAsDictionary options:0 error:&error];

          i++;  // MessageCounter


          // Update UI only sometimes
          if (i % 100)
          {

              dispatch_async(dispatch_get_main_queue(), ^
              {
                  @autoreleasepool {

                      ... update progress bar ...

                  }
              });

          }

      }
  } // end for loop

  [outData writeToURL:outURL atomically:YES];
});

另外，我不会使用if (i % 100)来决定是时候更新进度条了。相反，我会用：

CFTimeInterval lastProgressUpdate = CACurrentMediaTime();

for (  ... ) {
  ...

  if (CACurrentMediaTime() - lastProgressUpdate > 0.02) { // 1/60th of a second. Which is the refresh rate of most LCD screens
    ... update progress bar ....

    lastProgressUpdate = CACurrentMediaTime()
  }
}