如何让p5.js运行Pix2Pix？

Question

我想试试p5.js的ml5.js Pix2Pix示例。如果我只是复制代码，更新路径，并尝试让它在我的本地服务器上运行，它就不能工作。

这里也是如此：

​

// Copyright (c) 2019 ml5
//
// This software is released under the MIT License.
// https://opensource.org/licenses/MIT

/* ===
ml5 Example
Pix2pix Edges2Pikachu example with p5.js using callback functions
This uses a pre-trained model on Pikachu images
For more models see: https://github.com/ml5js/ml5-data-and-training/tree/master/models/pix2pix
=== */

// The pre-trained Edges2Pikachu model is trained on 256x256 images
// So the input images can only be 256x256 or 512x512, or multiple of 256
const SIZE = 256;
let inputImg, inputCanvas, outputContainer, statusMsg, pix2pix, clearBtn, transferBtn, modelReady = false,
  isTransfering = false;

function setup() {
  // Create a canvas
  inputCanvas = createCanvas(SIZE, SIZE);
  inputCanvas.class('border-box').parent('canvasContainer');

  // Display initial input image
  inputImg = loadImage('https://ml5js.github.io/ml5-examples/javascript/Pix2Pix/Pix2Pix_promise/images/input.png', drawImage);

  // Selcect output div container
  outputContainer = select('#output');
  statusMsg = select('#status');

  // Select 'transfer' button html element
  transferBtn = select('#transferBtn');

  // Select 'clear' button html element
  clearBtn = select('#clearBtn');
  // Attach a mousePressed event to the 'clear' button
  clearBtn.mousePressed(function() {
    clearCanvas();
  });

  // Set stroke to black
  stroke(0);
  pixelDensity(1);

  // Create a pix2pix method with a pre-trained model
  pix2pix = ml5.pix2pix('https://github.com/ml5js/ml5-library/blob/main/examples/p5js/Pix2Pix/Pix2Pix_callback/models/edges2pikachu.pict', modelLoaded);
}

// Draw on the canvas when mouse is pressed
function draw() {
  if (mouseIsPressed) {
    line(mouseX, mouseY, pmouseX, pmouseY);
  }
}

// Whenever mouse is released, transfer the current image if the model is loaded and it's not in the process of another transformation
function mouseReleased() {
  if (modelReady && !isTransfering) {
    transfer()
  }
}

// A function to be called when the models have loaded
function modelLoaded() {
  // Show 'Model Loaded!' message
  statusMsg.html('Model Loaded!');

  // Set modelReady to true
  modelReady = true;

  // Call transfer function after the model is loaded
  transfer();

  // Attach a mousePressed event to the transfer button
  transferBtn.mousePressed(function() {
    transfer();
  });
}

// Draw the input image to the canvas
function drawImage() {
  image(inputImg, 0, 0);
}

// Clear the canvas
function clearCanvas() {
  background(255);
}

function transfer() {
  // Set isTransfering to true
  isTransfering = true;

  // Update status message
  statusMsg.html('Applying Style Transfer...!');

  // Select canvas DOM element
  const canvasElement = select('canvas').elt;

  // Apply pix2pix transformation
  pix2pix.transfer(canvasElement, function(err, result) {
    if (err) {
      console.log(err);
    }
    if (result && result.src) {
      // Set isTransfering back to false
      isTransfering = false;
      // Clear output container
      outputContainer.html('');
      // Create an image based result
      createImg(result.src).class('border-box').parent('output');
      // Show 'Done!' message
      statusMsg.html('Done!');
    }
  });
}

<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/p5.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/addons/p5.dom.min.js"></script>
<script src="https://unpkg.com/ml5@latest/dist/ml5.min.js" type="text/javascript"></script>

<h1>Pix2Pix Edges2Pichaku Example</h1>
<p>1. Wait until the model is loaded</p>
<p>2. Press your mouse to draw a Pikachu on the left side of the canvas.</p>
<p>3. A colored Pikachu image will automatically appear on the right side of the canvas in ~2 seconds. You could also click the "Transfer" button to generate an new image.</p>
<p>4. You could click the "Clear" button to clear the canvas and draw again.</p>
<p id="status">Loading Model... Please wait...</p>
<div class="flex">
  <div>
    <div id="canvasContainer"></div>
    <div id="btnContainer" class="flex flex-space-between">
      <button id="clearBtn">Clear</button><br />
      <button id="transferBtn" class="btn">Transfer</button>
    </div>
  </div>
  <div id="transferContainer">
  </div>
  <div id="output"></div>
</div>

​

这里还有一个jsFiddle：https://jsfiddle.net/L6oaydrm/

有谁知道怎么让它运行吗？会很感激的。

Answer 1

我想我能够让“回调”的例子在本地使用，只需做一些修改：

1. 从示例下载文件：回调
2. 调整index.html以从代码中的ml5.min.js URL加载unpkg.com。
3. 创建一个新功能：

function startTransfer(){
  // Create a pix2pix method with a pre-trained model
  pix2pix = ml5.pix2pix('./models/edges2pikachu.pict', modelLoaded);
}

1. 将对transfer()的所有调用( modelLoaded()中的第一次调用除外)替换为startTransfer()。
2. 启动一个简单的本地web服务器；对我来说：python -m http.server成功了。

这个例子似乎奏效了。我可以在画布上画画，而ML模型将在我添加的新行中重新绘制Pikachu图像因子。注意，有时初始传输在加载模板图像(input.png)之前运行，结果是一个乱七八糟的黄色/红色像素；单击“transfer”可以修复这个问题。

基本上，它总是将模型重新加载到ml5库中；我不知道它对性能的影响，但它在浏览器中重新绘制的速度相对较快。该文件将在浏览器中缓存，因此这并不是一个问题，但我不确定ml5.jslib的内部结构以及ml5.pix2pix(...)的功能。

我已经将修改过的代码(包括对JS的其他一些调整)放在https://jsfiddle.net/lecrte/jvohcw8r/16/上..。但是它不能在那里工作，因为相对于HTML，资产是不可用的，而且由于CORS问题，我们不能直接从edges2pikachu.pict加载github.com。