Amazon S3 and Cloudfront cache, how to clear cache or synchronize their cache

cloudfront versioning
invalidate cloudfront cache console
aws delete invalidation
s3 cache
cloudfront purge tool
s3 bucket cloudfront cache
invalidate cloudfront url
cloudfront invalidation regex

I have a cron job that runs every 10 minutes and updates the content-type and x-amz-meta. But since yesterday it seems like after the cron job run, Amazon is not picking up the changes made (refreshing his cache).

I even went and made the changes manually but no change...

When a video is uploaded it has a application/x-mp4 content-type and the cron job changes it to video/mp4.

Although S3 has the right content type video/mp4 cloudfront shows application/x-mp4(old content-type) ....

The cron job has been working for the last 6 months without a problem.

What is wrong with amazon caching? How can i synchronize the caching?

Use Invalidations to clear the cache, you can put the path to the files you want to clear, or simply use wild cards to clear everything.

http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html#invalidating-objects-api

This can also be done using the API! http://docs.aws.amazon.com/cloudfront/latest/APIReference/API_CreateInvalidation.html

The AWS PHP SDK now has the methods but if you want to use something lighter check out this library: http://www.subchild.com/2010/09/17/amazon-cloudfront-php-invalidator/

user3305600's solution doesn't work as setting it to zero is the equivalent of Using the Origin Cache Headers.

Is there a way to force cache refresh, By default, CloudFront caches a response from Amazon S3 for 24 hours an S3 object to remove it from the CloudFront distribution's cache. But since yesterday it seems like after the cron job run, Amazon is not picking up the changes made (refreshing his cache). I even went and made the changes manually but no change When a video is uploaded it has a application/x-mp4 content-type and the cron job changes it to video/mp4 .

As to the actual code

get your CloudFront distribution id

aws cloudfront list-distributions

Invalidate all files in the distribution, so CloudFront fetches fresh ones

aws cloudfront create-invalidation --distribution-id=S11A16G5KZMEQD --paths /

My actual full release script is

#!/usr/bin/env bash

BUCKET=mysite.com
SOURCE_DIR=dist/

export AWS_ACCESS_KEY_ID=xxxxxxxxxxx
export AWS_SECRET_ACCESS_KEY=xxxxxxxxx
export AWS_DEFAULT_REGION=eu-west-1


echo "Building production"
if npm run build:prod ; then
   echo "Build Successful"
else
  echo "exiting.."
  exit 1
fi


echo "Removing all files on bucket"
aws s3 rm s3://${BUCKET} --recursive


echo "Attempting to upload site .."
echo "Command:  aws s3  sync $SOURCE_DIR s3://$BUCKET/"
aws s3  sync ${SOURCE_DIR} s3://${BUCKET}/
echo "S3 Upload complete"

echo "Invalidating cloudfrond distribution to get fresh cache"
aws cloudfront create-invalidation --distribution-id=S11A16G5KZMEQD --paths / --profile=myawsprofile

echo "Deployment complete"  
References

http://docs.aws.amazon.com/cli/latest/reference/cloudfront/get-invalidation.html

http://docs.aws.amazon.com/cli/latest/reference/cloudfront/create-invalidation.html

Push Updated Amazon S3 Content from CloudFront, Shouldn't the "Default TTL" setting refresh the cloudfront cache every process physically remove the invalidated object from the s3 bucket? 5. Go to your Cloudfront distribution > Click invalidations tab. Type folder or a specific image which you want to invalidate. That’s it…. Cloudfront will invalidate this content and grab the fresh new files from the S3 bucket and then refresh and cache it across their networks.

S3 is not used for real time development but if you really want to test your freshly deployed website use

http://yourdomain.com/index.html?v=2
http://yourdomain.com/init.js?v=2

Adding a version parameter in the end will stop using the cached version of the file and the browser will get a fresh copy of the file from the server bucket

cloudfront cache clearance, The process I follow here as following: 1. Deploy latest files to S3. 2. Create cache invalidation request to clear cache of index.html. 3. Clear  I would like to know if it's possible to clear CloudFront's cache, The file concerned has changed on Amazon S3 but it's not being updated on CloudFront. Stack Exchange Network Stack Exchange network consists of 177 Q&A communities including Stack Overflow , the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Cloudfront will cache a file/object until the cache expiry. By default it is 24 hrs. If you have changed this to a large value, then it takes longer.

If you anytime needs to force clear the cache, use the invalidation. It is charged separately.

Another option is to change the URL (object key), so it fetches the new object always.

CloudFront cache invalidation not, These two commands can do it. To deploy the code: aws s3 sync ./ s3://bucket-​name-here/ --delete. Create an invalidation in CloudFront:. Or try adding a query parameter to the end of the URL. Browsers cache by URI not file, so if it's a browser cache problem, that's one way to quickly identify it. Also, are you using S3 or is it frontend by CloudFront (or whatever their CDN is called?) I've heard there can be server cache issues with that service. – jmort253 May 20 '12 at 19:58

If you're looking for a minimal solution that invalidates the cache, this edited version of Dr Manhattan's solution should be sufficient. Note that I'm specifying the root / directory to indicate I want the whole site refreshed.

export AWS_ACCESS_KEY_ID=<Key>
export AWS_SECRET_ACCESS_KEY=<Secret>
export AWS_DEFAULT_REGION=eu-west-1

echo "Invalidating cloudfrond distribution to get fresh cache"
aws cloudfront create-invalidation --distribution-id=<distributionId> --paths / --profile=<awsprofile>

Region Codes can be found here

You'll also need to create a profile using the aws cli. Use the aws configure --profile option. Below is an example snippet from Amazon.

$ aws configure --profile user2
AWS Access Key ID [None]: AKIAI44QH8DHBEXAMPLE
AWS Secret Access Key [None]: je7MtGbClwBF/2Zp9Utk/h3yCo8nvbEXAMPLEKEY
Default region name [None]: us-east-1
Default output format [None]: text

How do you invalidate cache of index.html for a static site hosted on , Adding Headers to Your S3 Objects To Control Cache. Say we have a aws s3 sync --delete . s3://my_website.com --acl public-read. The  cloudfront purge tool which is a chrome extension can also be used to (invalidate files ) clear amazon cloudfront cache. You can find more info at Cloudfront Purge Tool

This is How I Reduced My CloudFront Bills by 80% - FAUN, Speeding Up Your Amazon S3 Based Website Using Amazon CloudFront 14 Controlling How Long Amazon S3 Content is Cached by Amazon command recursively uploads files and synchronizes your Amazon S3 bucket with your Lifecycle policy #2: After Y days, automatically delete logs from Amazon S3. Glacier. If the files are in the cache, CloudFront forwards the files to the POP that requested them. As soon as the first byte arrives from regional edge cache location, CloudFront begins to forward the files to the user. CloudFront also adds the files to the cache in the POP for the next time someone requests those files.

[PDF] Hosting Static Websites on AWS, AWS is ready to serve cached content from a bucket; now I need to get and set a no-cache header; run aws s3 sync --delete to delete the old,  For web distributions, you can use Cache-Control or Expires headers, and CloudFront minimum, maximum, and default TTL values to control the amount of time in seconds that CloudFront keeps an object in the cache before forwarding another request to the origin. Header values also determine how long a browser keeps an object in the cache before

Deploying Ember apps to S3 and Cloudfront, get your CloudFront distribution id. aws cloudfront list-distributions. Invalidate all files in the distribution, so CloudFront fetches fresh ones. aws cloudfront  Amazon S3 and Cloudfront cache, how to clear cache or synchronize their cache Basically you need to clear the cache on S3. share | improve this answer edited May 23 '17 at 12:14

Comments
  • Maybe select an answer ;-)
  • This is indeed a better answer
  • That library looks to be outdated.
  • @PhilTune Post was from 2014
  • Thanks for posting this--I needed to invalidate a distribution to fully rid my app of an old, overwritten file version that kept popping up.
  • Instead of deleting all files in the bucket first, you can use the --delete option of aws s3 sync to do that for you. i.e. aws s3 sync ${SOURCE_DIR} s3://${BUCKET}/ --delete
  • Worthwhile to mention here, at path use "/*" invalidate all files in bucket and "/folder_name/*" to invalidate files in folder name. also I think it costs money
  • Your solution doesn't work as setting it to zero is the equivalent of using the Origin Cache Headers.
  • Sorry! Invalidation is the only way to do it! I guess
  • 1000 invalidations for free. I guess that's not expensive ;)
  • What do you mean they can't be reverted? Doesn't the cache get filled again upon the next request?
  • First of all, don't confuse a CDN with a cache. Second, no sorry, I didn't mean this, I meant that you can't cancel the invalidation once you create it, but it does apply only to the existing objects in the distribution.
  • Anyway ... one year after my comment, I'm not completely agree with myself, to be honest.
  • What does invalidate cache actually do? I have it in my pipeline but I always have to hard reload my website to see the changes. I use s3 sync to update website files on S3. I think invalidate cache only clears files from Cloudfront Edge location but not the browsers cache.