从EC2到S3的MIgrating数据？

Question

我正在运行一个Rails应用程序，使用Paperclip来处理文件附件和图像大小调整等。该应用程序目前托管在EngineYard云上，所有附件都存储在他们的EBS中。考虑使用S3来处理所有的回形针附件。

有没有人知道一种既好又安全的迁移方法？非常感谢！

Answer 1

您可以创建一个rake任务，该任务迭代您的附件并将每个附件推送到S3。我之前在attachment_fu上用过这个--不会有太大的不同。这使用了aws-s3 gem。

基本上过程是: 1.从数据库中选择需要移动的文件2.将其推送到S3 3.更新数据库以反映该文件不再存储在本地(这样您可以批量操作，无需担心将同一文件推送两次)。

@attachments = Attachment.stored_locally
@attachments.each do |attachment|

  base_path = RAILS_ROOT + '/public/assets/'
  attachment_folder = ((attachment.respond_to?(:parent_id) && attachment.parent_id) || attachment.id).to_s
  full_filename = File.join(base_path, ("%08d" % attachment_folder).scan(/..../), attachment.filename)
  require 'aws/s3'

  AWS::S3::Base.establish_connection!(
    :access_key_id        => S3_CONFIG[:access_key_id],
    :secret_access_key    => S3_CONFIG[:secret_access_key]
  )

  AWS::S3::S3Object.store(
    'assets/' + attachment_folder + '/' + attachment.filename,
    File.open(full_filename),
    S3_CONFIG[:bucket_name],
    :content_type => attachment.content_type,
    :access => :private
  )

  if AWS::S3::Service.response.success?
    # Update the database
    attachment.update_attribute(:stored_on_s3, true)

    # Remove the file on the local filesystem
    FileUtils.rm full_filename

    # Remove directory also if it is now empty
    Dir.rmdir(File.dirname(full_filename)) if (Dir.entries(File.dirname(full_filename))-['.','..']).empty?
  else
    puts "There was a problem uploading " + full_filename
  end
end

Answer 2

我发现自己也处于同样的情况，于是我采用了bensie的代码，并让它为我自己工作--这就是我想出来的：

require 'aws/s3'

# Ensure you do the following:
#   export AMAZON_ACCESS_KEY_ID='your-access-key'
#   export AMAZON_SECRET_ACCESS_KEY='your-secret-word-thingy'
AWS::S3::Base.establish_connection!


@failed = []
@attachments = Asset.all # Asset paperclip attachment is: has_attached_file :attachment....
@attachments.each do |asset|
  begin
    puts "Processing #{asset.id}"
    base_path = RAILS_ROOT + '/public/'
    attachment_folder = ((asset.respond_to?(:parent_id) && asset.parent_id) || asset.id).to_s
    styles = asset.attachment.styles.keys
    styles << :original
    styles.each do |style|
      full_filename = File.join(base_path, asset.attachment.url(style, false))


      AWS::S3::S3Object.store(
        'attachments/' + attachment_folder + '/' + style.to_s + "/" + asset.attachment_file_name,
        File.open(full_filename),
        "swellnet-assets",
        :content_type => asset.attachment_content_type,
        :access => (style == :original ? :private : :public_read)
      )

      if AWS::S3::Service.response.success?        
        puts "Stored #{asset.id}[#{style.to_s}] on S3..."
      else
        puts "There was a problem uploading " + full_filename
      end
    end
  rescue
    puts "Error with #{asset.id}"
    @failed << asset.id
  end
end

puts "Failed uploads: #{@failed.join(", ")}" unless @failed.empty?

当然，如果您有多个模型，您将需要根据需要进行调整...