Amazon AWS

[Zetetic]

Ah. It's depressing how much that sort of thing isn't strongly discouraged by various application/deployment frameworks.[nb]Rails has only recently had a go at tackling this by default, and its implementation isn't that helpful.[/nb]

Not sure if either of these would've helped but it's worth knowing that:

• AWS allows you to define keys with particular permissions. (Although not always with enough granularity even in my limited experience.)
• AWS supports OTP.

[biggytitbo]

Wonder what the odds of amazon refunding the money is?

[biggytitbo]

The amazon console is just shite too. There's no way of me knowing that was happening without having to manually trawl though all the different regions, something id have no reason to do normally.

[hedgehog90]

I was right to be worried about hidden charges using this. I've had the 'free tier' ec2 instance running for 2 weeks and they've charged me $294!


Clearly the have a different definition of free to me, the bastards.

I set up an EC2 instance just the other day out of curiousity, to go along with my S3 bucket.
Almost immediately after setting the thing up I realised it wasn't what I wanted, and I'd be better off just using a cheap hosting service like Hostgator.
I terminated the instance, but I want to be certain that it's truly gone. After reading about your troubles now I'm worried there might be some hidden charges coming my way.
It shows Running Instances as 0, but I'm still worried. Should I be?

EDIT - And I'm also worried about S3... I'm still in the free usage tier, but if I use say, 500GB of bandwidth and 10 million 'gets' in a month how much will I be charged? I've got it down as $15 + $4.
Does anybody know of any hidden charges with S3 that I should be aware of?

[Zetetic]

I think the EC2 problem for biggy is specific to either creating an extra instance by mistake (and forgetting about it) or handing out his authentication details for anyone to have a go on his account.

(The console certainly doesn't help you avoid this things...)



I'd take a look at the calculator - http://calculator.s3.amazonaws.com/index.html - for the S3 costs. Storage isn't bandwidth - 500GB/month out of AWS will cost you a fair bit.

Other - S3-compatible - storage providers are available ( https://www.greenqloud.com/storageqloud/ ) that may have the advantage of being cheaper (depending), not run by Amazon and not hosted in the United States.

[biggytitbo]

I set up an EC2 instance just the other day out of curiousity, to go along with my S3 bucket.
Almost immediately after setting the thing up I realised it wasn't what I wanted, and I'd be better off just using a cheap hosting service like Hostgator.
I terminated the instance, but I want to be certain that it's truly gone. After reading about your troubles now I'm worried there might be some hidden charges coming my way.
It shows Running Instances as 0, but I'm still worried. Should I be?

EDIT - And I'm also worried about S3... I'm still in the free usage tier, but if I use say, 500GB of bandwidth and 10 million 'gets' in a month how much will I be charged? I've got it down as $15 + $4.
Does anybody know of any hidden charges with S3 that I should be aware of?

It's just a constant worry using them since you're unable to cap how much you pay you can never be 100% sure you aren't going to be whacked with a nasty bill. If you could actually cap the usage levels rather than it been completely open ended then it would be a good service, but you can't so it isn't.

[Zetetic]

You can set billing alerts (and indeed daily reports), although again it's not a very friendly interface - https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/monitor_estimated_charges_with_cloudwatch.html

[hedgehog90]

url="http://calculator.s3.amazonaws.com/index.html"]http://calculator.s3.amazonaws.com/index.html[/url] - for the S3 costs. Storage isn't bandwidth - 500GB/month out of AWS will cost you a fair bit.
Other - S3-compatible - storage providers are available ( https://www.greenqloud.com/storageqloud/ ) that may have the advantage of being cheaper (depending), not run by Amazon and not hosted in the United States.

Storage wise it'll be tiny, no more than 1 GB so that's not a problem.
You just struck on something though that has me concerned - I set it up in the Ireland region, but 90% of traffic will be coming from the US. What does this mean? Basically I just don't get the whole region thing whatsoever.

You can set billing alerts (and indeed daily reports), although again it's not a very friendly interface - https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/monitor_estimated_charges_with_cloudwatch.html

I've just done that, thanks.

[biggytitbo]

Pretty impressed with Amazons customer service over this whole debacle. The couldn't have been more helpful all-round and have refunded me the entire amount plus a bit extra.


The can keep up their anti-competitive practises all they like if they're going to be this helpful.

[hedgehog90]

This isn't really the right place to ask but... whatever, I'm hoping someone will understand my question and point me in the right direction.

I'm using S3 in a project made with Flash. The application downloads images from the S3 Bucket, I have a working crossdomain policy file which allows communication between Flash and S3 and it works just fine, but now I need to upload images to the bucket itself.
I found a library that uses the HTTP POST method to send the image, however, this requires my AWS access key and secret access key.
Here's the problem - I need to write these in the code, and Flash can be easily decompiled and these details can be obtained by ne'er do wells.
Also, the URL request that's sent contains my access key and a base64 encoded signed policy string which contains my secret access key, so if the user is using a debug tool I'm pretty sure they can listen for these and get both keys.

I could use PHP on my webserver that receives the image from Flash and then uploads it to S3, but this puts unnecessary strain on my webserver which I'd much rather avoid. It's just a cheap hostgator site and it's already straining under the current load.

Also, I can't use sockets because Flash is a right bastard when it comes to security measures. I would require a socket policy file server or something and that apparently isn't possible with S3. Also this means I can't track the progress of the upload itself, the only signal I get is when there's an error or when it finishes... so I've just got a barber's pole to signify something is working saying "Uploading..." with a message telling the user to "Be patient while it uploads"

So my question is, how do you protect yourself from hackers in a situation like this? My application can be easily decompiled and it contains vital information that would effectively allow anyone to login to my S3 and delete everything. Even if I make it unintelligible in the code (obfuscation?), there are tools out there that can listen for the URL Request which contains all my info when it's sent.

[biggytitbo]

My mistake was including my key file in my git repo, which obviously some 'git' was scanning for berks like me. It's a very bad idea to include it in anywhere in your front end code, just don't do it. The only way round it is to move the request to a server, and if you're on the free tier you could use aws itself for that for nothing.

[Zetetic]

You should - I think - create an 'AWS IAM User' whose policy dictated that all they could do was read and write a particular S3 bucket (and I believe you could control that further with S3 ACLs to prevent the 'delete everything' scenario).[nb]Here's an example of such a policy, apparently.[/nb]

That's still not enough, I'd suggest, to make the credentials openly available - someone will still be able to fill your S3 bucket with huge amounts of child abuse imagery, fairly quickly I'd imagine. I think you will need server-side code to do proper damage control - validation they they're actually images, limiting the rate of uploads from a single visitor perhaps, and so on.

if you're on the free tier you could use aws itself for that for nothing.

Although I think you will incur bandwidth costs, although I'm guessing that they wouldn't be very large.

[hedgehog90]

The only way round it is to move the request to a server, and if you're on the free tier you could use aws itself for that for nothing.

You mean the thing that got you in a lot of bother? I'm not sure I trust it. Also I found it terribly confusing and I felt out of my zone when I was setting it up, basically I just want an environment which will run a php with relative ease. If thousands of people per month were running this php which took requests (ie, received image files no larger than 1.5MB from user) would that be expensive? My worry is that even if it is cheap it could get out of hand.

Also, I've just reminded myself of the question I asked but was never answered, what's with regions? What's their importance? I have an Irish Bucket. Is that any different to an American Bucket? Is AWS a bit racist towards some buckets but not others? What if my Irish bucket has mostly Americans downloading its contents? I'm sure this is easily googleable but I'd prefer to hear it from one of you experts.

You should - I think - create an 'AWS IAM User' whose policy dictated that all they could do was read and write a particular S3 bucket (and I believe you could control that further with S3 ACLs to prevent the 'delete everything' scenario).[nb]Here's an example of such a policy, apparently.[/nb]

That's still not enough, I'd suggest, to make the credentials openly available - someone will still be able to fill your S3 bucket with huge amounts of child abuse imagery, fairly quickly I'd imagine. I think you will need server-side code to do proper damage control - validation they they're actually images, limiting the rate of uploads from a single visitor perhaps, and so on.
Although I think you will incur bandwidth costs, although I'm guessing that they wouldn't be very large.

I've got a way of limiting bandwidth abuse in the Flash application. Also I've just implemented the IAM user thing so that the access key can only do Put actions, tah for that!
That's 50% of my problem solved :)

[biggytitbo]

You can run a linux micro instance 24/7 for a year for nothing, that should do everything you need.

[hedgehog90]

Right. So I've set up a new EC2 instance and got it working as a web server.
So if I set up a php script that handles the files and puts them on S3 there shouldn't be a problem usage wise?
Also, you say this won't cost a penny if I have it running 24/7 for a year, what about after a year?
What if my php handler was receiving on average between 1 and 2 GB every day?

Also, I did a bit of research and can't find any discernible difference between regions. Some are more stable and have more features than others, that's about it.

EDIT: Also, is there any way to stop the instance if it gets out of hand without losing all my data?

[biggytitbo]

Right. So I've set up a new EC2 instance and got it working as a web server.
So if I set up a php script that handles the files and puts them on S3 there shouldn't be a problem usage wise?
Also, you say this won't cost a penny if I have it running 24/7 for a year, what about after a year?
What if my php handler was receiving on average between 1 and 2 GB every day?

Also, I did a bit of research and can't find any discernible difference between regions. Some are more stable and have more features than others, that's about it.

It's free to use for a year, but you might still pay a bit if you go over the bandwidths limits or number of requests. It's very cheap though, unless you have shitloads of users?

[hedgehog90]

What's "very cheap"? And what if it's a unique user every time?
I'm just trying to work out if I should use this or my old shitty webserver.