express中的multer-s3上传速度慢得离谱

Question

在我的express应用中，通过multer-s3上传文件到S3的速度慢得离谱(200kb需要10秒，延迟与文件大小相关)。问题是multer-s3，但有人能告诉我如何进一步调试库中发生的事情吗？不再支持将onFileUploadStart等传递给multer。我不知道multer-s3在做什么，以及延迟是在哪个阶段导致的。

问题不是亚马逊网络服务的S3，因为从命令行上传超快，也不是更多，因为上传到本地磁盘也很快。任何直接的S3 javascript操作也可以像预期的那样快速工作。

我试过关闭multer的内容自动输入功能，但效果不是很好。

var upload = multer({
  storage: multerS3({
    s3: s3,
    bucket: bucketName,
    acl: 'public-read',
    contentType: multerS3.AUTO_CONTENT_TYPE,
    metadata: function (req, file, cb) {
      cb(null, {fieldName: file.fieldname});
    },
    key: function (req, file, cb) {
      cb(null, Date.now().toString() + "-" + file.originalname);
    }
  },)
})

router.post('/', upload.array('files'), function(req, res, next) {
  console.log('Successfully uploaded ' + req.files.length + ' files!');
});

Want to have sub-second latency as I do with direct AWS CLI upload or using multer to store to local disk, takes 10 seconds to minute or more on sub-MB files.

Answer 1

获得洞察力的一种方法是在响应套接字上代理write函数，并检查它是如何被调用的：

var upload = multer({
  storage: multerS3({
    s3: s3,
    bucket: bucketName,
    acl: 'public-read',
    contentType: multerS3.AUTO_CONTENT_TYPE,
    metadata: function (req, file, cb) {
      cb(null, {fieldName: file.fieldname});
    },
    key: function (req, file, cb) {
      cb(null, Date.now().toString() + "-" + file.originalname);
    }
  },)
})

router.post('/'
(req, res, next) => { // debug middleware
  const write = res.socket.write;
  res.socket.write = (function(...args) {
    // inspect args, look at the time between invocations of write
    // also check the size of the chunks being passed to write
    return write(...args)
  }).bind(res.socket);
  next();
}
, upload.array('files'), function(req, res, next) {
  console.log('Successfully uploaded ' + req.files.length + ' files!');
});