?

Log in

No account? Create an account
Amazon Web Services - Chronicles of a Hereditary Geek [entries|archive|friends|userinfo]
Darth Paradox

[ website | Pyrlogos - a fantasy webcomic ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Amazon Web Services [Nov. 22nd, 2006|03:34 pm]
Darth Paradox
[Tags|]
[Current Location |work]

Next week, I'll be taking a short training course in developing with Amazon Web Services, followed by a few days of work developing a small application using AWS.

My question to all of you is: Given the vast quantities of data Amazon stores, what do you wish you could do with it that you can't right now?

Update for clarification: I am not in a position to change the way the Amazon site itself works. What I'm looking at is new ways of surfacing/processing/etc the existing data and functionality that the Web Services provide. (For example, I could create a separate page that manipulates some set of recommendations provided to me by AWS, but I couldn't actually change what Amazon itself recommends.)
linkReply

Comments:
[User Picture]From: kinkyprude
2006-11-23 12:18 am (UTC)
Build a TARDIS. That, and I'm disapointed that I can't digitize my consciousness and live on the internet, and I think Amazon should provide this service.
(Reply) (Thread)
[User Picture]From: darthparadox
2006-11-23 12:33 am (UTC)
Well, with S3, we're capable of storing vast amounts of data for you, and EC2 allows us to spin up as many virtual servers as we need to run the processes. So if you figure out how to upload yourself into a server farm, we'd be happy to host it for you. :D
(Reply) (Parent) (Thread)
(Deleted comment)
(Deleted comment)
[User Picture]From: unspeakablevorn
2006-11-23 01:28 am (UTC)
I would like it to make it so I don't get (for instance) Diablo II showing up in my "recommended for you" list if I say I own Diablo Battle Chest. But that's more on the store side. I don't know. I don't really see Amazon as a... a... web services kind of thing.

Vorn
(Reply) (Thread)
[User Picture]From: fiercekitty
2006-11-23 01:32 am (UTC)
(Reply) (Parent) (Thread)
[User Picture]From: darthparadox
2006-11-23 01:54 am (UTC)
Yeah, I have no idea how the nitty-gritty of the recommendations engine works. I think a lot of it, compares what different choices people look at when tending to buy something, to try to create an idea of "related items"... but in the case where the choice is an exclusive one (someone will buy the Battle Chest or Diablo II, but never (or extremely rarely) both), I don't think the engine can recognize the fact that the relationship between desire to buy those items is actually an inverse one.

Now that I've thought it through, though, having an inverse-recommendation engine wouldn't be overly complex. I'll have to see if I can forward the idea to the Recommendations guys.
(Reply) (Parent) (Thread)
[User Picture]From: fiercekitty
2006-11-23 01:30 am (UTC)
Not sure if you work on this, but recommendations could be better :P
(Reply) (Thread)
[User Picture]From: darthparadox
2006-11-23 01:55 am (UTC)
I don't, but I think I could forward the idea (in my response to Vorn's comment) to the Recommendations guys. I'll see what I can do.
(Reply) (Parent) (Thread)
[User Picture]From: xaandria
2006-11-23 02:26 am (UTC)
Make it so that if I'm buying something as a gift, I can check a "buying this as a gift" box. So that when Amazon recommends things to me, I'm not getting Spanish versions of "The Very Hungry Caterpillar" just because I bought a Spanish children's book for Child's Play last year. Or "How to Draw Manga." Or dozens of other things that it thinks I like because I bought it. I didn't buy it for meeeee...
(Reply) (Thread)
[User Picture]From: xaandria
2006-11-23 02:27 am (UTC)
And yes, I know you don't do recommendations. I just wanted to whine.
(Reply) (Parent) (Thread)
[User Picture]From: kikiduck
2006-11-23 02:31 am (UTC)
Man, I feel like I should be able to come up with something for this, but I'm drawing a complete blank. =(
(Reply) (Thread)
[User Picture]From: darthparadox
2006-11-23 02:45 am (UTC)
Yeah, really, you're the one studying information science.
(Reply) (Parent) (Thread)
[User Picture]From: kikiduck
2006-11-23 03:28 am (UTC)
Ask me again in two years? =P
(Reply) (Parent) (Thread)
From: kernighan
2006-11-23 04:50 pm (UTC)

happy thanksgiving!

DESTROY IT!!!!

I have concerns about the large amounts of data Amazon stores. I wish that the default was to not store any data. Currently, it doesn't *ever* log you out unless you explicitly say so, and it stores all your purchases and even the sites you look at. This is unethical: a site shouldn't store any data unless you explicitly request it to, and there should be an easily accessible way to delete all stored data pertaining to you.

I know this is probably beyond your ability, but still.

And yeah, I am curious about the IBM suit. My guess is that they sued in an attempt to show how silly patents were (I can't imagine a patent on online advertising being enforceable, as it's way too obvious of an idea, and the timing is awfully convenient, as the patent expires in a few months anyway.)
(Reply) (Thread)
From: dacut
2006-11-24 08:46 am (UTC)

Re: happy thanksgiving!

We don't store sites you look at, unless you mean the pages on the site.

About logging out... yeah, the problem is there's no sense of being "logged in" to the site, which makes logging out difficult. I work with the guy who's responsible for authentication and is revamping the service behind it. There will be a concept of revoking authentication, but designing the UI behind it in an understandable way will be the big task.

We are required by U.S. law to store your transaction history (to be able to prove we're not cooking the books, for example). Any U.S. company which doesn't is violating the Sarbanes-Oxley (SOX) act, which was enacted after Enron.

Ironically, we're also required by European law to delete all personal information upon request. (How we manage to satisfy that conundrum is rather interesting and involves a lot of crazy hackery.) This isn't self-service, though -- the number of customer service contacts we'd get requesting us to "undelete" information (for warranty receipts, etc.) would be unbearable.

I disagree with your assertion that it's unethical. Forcing people to jump through hoops to get the current site functionality would be a huge step backwards in usability and render it useful only for geeks like me and those persistent enough to toggle through a thousand different options. The privacy policy is taken extremely seriously; any site features which store information undergo a review with other developers and the legal team.
(Reply) (Parent) (Thread)
From: kernighan
2006-11-28 01:12 am (UTC)

but what color sox?

Yeah, I did mean pages on the site. It's a little creepy that the site stores all the on-site pages a given account looks at.

There's a visible display when I'm logged in or not. It says "Hello, Jeremy. If you're not Jeremy, click here." Clicking there does log me out: however, I wish that it logged me out after, say, 15 minutes, rather than staying logged in forever. I'm glad to note that there is indeed a sign-out link from the main page. There used to not be one.

Thanks for the info on the SOX act. I can see how it would be necessary to store purchase history. Please explain the conundrum of satisfying SOX and Eurolaw at the same time. That sounds interesting, and by "interesting" I mean "impossible". yay crazy hackery!

I don't think jumping through hoops is necessary. Just have a setting for turning all "functionality" beyond what's required by law on or off. I personally don't find the recommendations, stored pages, etc. at all useful. They actually make it less likely I'll use the site. Store what's required by law, and have a "maximum privacy, minimum functionality" option for paranoid people like me.

This reminds me of when I called Sprint to cancel Call Waiting. The customer service rep asked why I didn't want that feature, and I said, "because I don't think it's a feature".
(Reply) (Parent) (Thread)
From: dacut
2006-11-28 05:31 am (UTC)
There are different levels of logged-in-ness. It's not a binary decision. I don't remember what the exact times are, but I think it's something like: recognition and personalization continues as long as your cookie exists and hasn't been revoked; account history if you signed in within the last 15 minutes; checkout if you signed in within the last 5 minutes; and certain other operations always insist on sign-in.

The sign out link is just axing your cookie.

I can't get into the details of the SOX-Eurolaw compliance, but it basically involves scrambling all of your personal information while keeping shipment manifests and payment transaction ids around. Essentially, our claim is, "A shipment of these goods occurred, and it was paid for. If you don't believe us, go talk to the shipping company and payment processor, and give them these IDs."

As for site functionality... I personally agree with you. In my case, it's more a matter of clutter than privacy. Unfortunately, I doubt we'll ever design a site that isn't geared toward what the majority finds useful (as measured by various hard metrics which notably do *not* include user feedback -- there's a big disconnect betweeen what people say they want and what they use). Fortunately, our web service APIs are open just enough to enable third-party features like Amazon Light. I suspect there will eventually come a day when the majority of Amazon customers use other sites to do their Amazon shopping.

Out of curiosity, why don't you like call waiting? (Or, rather, how does it reduce your privacy?)
(Reply) (Parent) (Thread)
From: kernighan
2006-11-30 07:57 pm (UTC)
Hmm, so you employees probably have uber-levels of logged-in-ness, and someone way up the hierarchical chart probably has access to buttons that can blow up Amazon's various offices.

I agree with you about the clutter. It's both the clutter and the privacy issues. You say amazon doesn't care about user feedback, just what people actually use? In other words, amazon doesn't respond to what its customers request? That's incredibly disturbing if it's true. *sells stock in Amazon* Seriously, ignoring customer feedback is a good way to lose customers. The way the site is set up, many site "features" are difficult or impossible not to use. Please explain more about these hard metrics.

Call waiting doesn't reduce privacy (that I know of... advanced paranoids can probably come up with some theory on how it might). The comparison was that they're both problems that are called "features".

Sprint users are automatically given call waiting, and it doesn't explicitly say anywhere you can get rid of it. But I called, and it turns out you can. I was amused that the customer service rep. thought it was a "feature": she said "Why don't you want this feature?".

I don't like call waiting because I think it's an annoyance. I don't like being interrupted while talking on the phone: I hate when people do that to me, and I wouldn't do it to anyone else.
(Reply) (Parent) (Thread)
From: dacut
2006-12-01 06:46 am (UTC)
There's no über login -- that's a mischaracterization of how authentication works. (In fact, the website and corporate systems are completely disparate -- Jeff Bezos' account is just like any other user's account.) Authentication is a measure of how confidently we believe you are who you claim to be.

What you're thinking of is authorization -- what are you allowed to access? As a retail website buyer, there aren't many bits anyone can toggle here. For sellers, it's a completely different story.


The metrics don't take user feedback into account -- that is, "people (say they) like it" isn't a reason to keep a feature. The data act on is how users behave. I actually agree with this methodology -- in my experience, users are good at expressing frustration, but generally bad at expressing what they want.

So we do pay attention to what our users want; we just feel that actions speak louder than words.

Alas, I can't talk about the details of how this is implemented. This is one of the few things that is expressly verboten to discuss (it's considered a trade secret).
(Reply) (Parent) (Thread)
From: dacut
2006-11-24 08:49 am (UTC)
Store my browser bookmarks and address book in a way that's easily accessible via Firefox and Thunderbird.

I've also given thought to using S3 as a universal home directory type of service. Alas, the lack of file locking makes this difficult to achieve. (And S3 is dangerously close to capacity, so I'm not sure I'd want to trust it at this point.)
(Reply) (Thread)
[User Picture]From: inprotest
2006-11-26 05:21 pm (UTC)

wish list

there should be a master "save" on a wish list. changing a bunch of priorities or something then realizing there's no master save is annoying. and it's tedious to update, save, update, save, etc. doesn't seem particularly difficult to do (even though i don't do this in my own job), but i'd appreciate it.
heh, i'm going to feel stupid if this has been available all along and i just haven't noticed.
(Reply) (Thread)
[User Picture]From: darthparadox
2006-11-26 07:32 pm (UTC)

Re: wish list

As it happens, every single one of those "save" buttons is a master save. I agree that the interface design doesn't indicate that, but I just tested it, and I assure you that's the case.
(Reply) (Parent) (Thread)