MetaSkills.net

Authenticated S3 GETs For Private Objects Using Paperclip

Posted On: November 23rd, 2009 by kencollins

Yea I know, I am probably the last person on earth that is just getting around to using Paperclip. To be honest, most of my file upload code was written way before Paperclip or even AttachementFu was ever conceived. And frankly, I do not do much social app coding on the side - so the need never came up. But that changed recently and I wanted a really really good way of leveraging AWS::S3 storage with the best local app security while maintaining tight control over the files.

So the Paperclip wiki has a few links that already dealt with some ways of protecting your app's attachments. One mentions a method I totally love called security through obscurity. It uses a secure random token as part of the filename which combined with the original filename and the id partition makes for great random URLs. The other is a great walk thru on how to use the :url option of paperclip to point access control back to your own application for your normal biz logic.

The problem I see with both of these methods is that they do not allow you to maintain app control past the final URL handoff/redirect. It also requires that your S3 bucket is public. For instance, if you were to use the s3_permissions => :private option of Paperclip, then that URL given to you by Paperclip is pretty much worthless. I knew AWS::S3 had authenticated GETs that generated an automatically expiring URL, but saw no way of accessing its features using the abstract Paperclip::Attachment object. So this is what I did.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

class MyDownload
  
  has_attached_file :attachment, 
                    :storage => :s3, :bucket => 'mybucket',
                    :s3_credentials => {...}, :s3_protocol => 'https', 
                    :s3_permissions => :private,
                    :path => lambda { |attachment| ":id_partition/#{attachment.instance.random_secret}/:filename" },
                    :processors => [:noop]
  
  before_validation_on_create :set_random_secret
  
  
  def attachment_url
    "#{self.class.tableize}/#{id}/#{attachment_file_name}"
  end
  
  def authenticated_s3_get_url(options={})
    options.reverse_merge! :expires_in => 10.minutes, :use_ssl => true
    AWS::S3::S3Object.url_for attachment.path, attachment.options[:bucket], options
  end
  
  private
  
  def set_random_secret
    self.random_secret = ActiveSupport::SecureRandom.hex(8)
  end
  
end

Let me walk you thru some of the highlights of that class, the general concept following is that we are going to use the best of both examples in security mentioned above. First, the secure token, that is what #set_random_secret will generate for each instance. The :path option for Paperclip uses a proc to make sure each instance uses that attribute in the string that will be later interpolated further down. You can also see how I use the id partition too. Next, I have added two public instance methods. The first is #attachment_url and it will need a bit of explaining.

Currently in Paperclip, if you use the :url option and your :storage is set to :s3, then it is ignored. This could totally be intentional. So in a typical setup where you want the file download running through your own access control, you wold have a :url option like this :url => '/:class/:id/:filename'. So this is what #attachment_url mimics, it simply gets around that shortcoming and points the download action back to your own controller. How that controller would work is beyond the scope of this article, see the resources section below for those links.

The last example method is #authenticated_s3_get_url which dips right on down to the AWS::S3 library to get the URL for the object in the bucket. AWS::S3's doc mention that it will automatically generate a secure GET url that expires in 5 minutes. However in my example, you can see where I am changing that to 10 minutes and forcing the HTTPS protocol. This would be the URL that your own controller would do the final redirect to. This URL is for your private objects in your S3 bucket and will only work for the amount of time you want it too! Meaning your app stays in complete control. Putting it all together one more time...

# A MyDownload instance.
>> dl = MyDownload.find(4)

# This is totally useless for private buckets/objects.
>> dl.attachment.url
=> "https://s3.amazonaws.com/mybucket/000/000/004/147681c16fddc1e5/private.pdf?1258989107"

# This is what you use in your own views.
>> dl.attachment_url
=> "my_downloads/4/private.pdf"

# Your controller would redirect to this secure GET.
>> dl.authenticated_s3_get_url
=> "https://s3.amazonaws.com/mybucket/000/000/004/147681c16fddc1e5/private.pdf?AWSAccessKeyId=0HJD3NS9CVWN2JV89K02&Expires=1258990967&Signature=8aWsq4o5gXfpIrRZyeETddnOeFw%3D"

What Is That Noop Processor

Good eye! Did you see that I have a processor called Noop in the has_attached_file declaration? The default processor in Paperclip is the Thumbnail processor, which no matter what calls the ImageMagick identify command to see if it can do something to the file. I did not want that or any processing, just simple attachments. So I created this simple processor that just straight returns the file object. I made a ticket on the Paperclip's issue page that hopefully would allow a :processors => false option one day that would do this as well. So maybe one day it'll be a feature.

1
2
3
4
5
6
7
8
9
10

module Paperclip  
  class Noop < Processor
    
    def make
      file
    end

  end
end

Resources

Karl

  HOMEPAGE  | November 23rd, 2009 at 02:18 PM
Karl Damn, that is timely. Just started working on the same this morning. Especially like the Noop processor.

Karl

  HOMEPAGE  | November 25th, 2009 at 09:00 PM
Karl Ken, I have not tested extensively, but it appears that if you omit :styles => {} from has_attached_file, then paperclip will do no post processing. I tried it uploading some PDF files and XLS files and it throws no errors. According to paperclip readme: "NOTE: Because processors operate by turning the original attachment into the styles, no processors will be run if there are no styles defined." So maybe you don't need the Noop processor?

Ken Collins

  HOMEPAGE  | November 27th, 2009 at 06:43 PM
Ken Collins @Karl You are entirely correct! I just tested this today. Thanks for the heads up.

Tom Walpole

  HOMEPAGE  | November 30th, 2009 at 02:41 PM
Tom Walpole I find it easier to just add a paperclip interpolation in an intiializer Paperclip.interpolates(:s3_authenticated_url) do |attachment, style| AWS::S3::S3Object.url_for(attachment.path(style), attachment.bucket_name, :use_ssl=>true, :expires_in=>3.minutes) end and then in the has_attached_file call use the options :s3_permissions=>'private', :url=>':s3_authenticated_url'

Ken Collins

  HOMEPAGE  | December 1st, 2009 at 07:55 PM
Ken Collins

FYI, I noticed this commit in paperclip which has exact results as mine, but still does not use the host alias. I may get around to adding a patch soon.