Nanobeepers

I've been developing a framework using EC2, S3, S3DFS (DFS = Distributed File System) along with SQLite to try and solve the database problem on Amazon Web Services.

S3DFS allows you to mount an S3 bucket to one or multiple EC2 instances as if it were a local filesystem. It works at the block level and has read/write caching - so is very fast. (Note: It's free to use for development, but otherwise requires a paid commercial license.)

Since SQLite uses flat files and has no database server or client, it is ideal for use on S3. SQLite only scales well to a point, so to get around its limitations I am separating each user into their own database (each database is a separate flat file). This can sometimes be a bad idea, but SQLite allows you to attach multiple databases together to essentially create a temporary master database where you can run queries across all the user databases at once. This will be handy for search indexing, site-wide stats, etc. In order to change the user databases I will have utilities to allow global scheme alterations.

So, by adding EC2 instances that are all attached to the same S3 bucket, and using round-robin dns to distribute traffic among them, you can theoretically scale a web service infinitely. Just launch a new EC2 instance and add it to the round-robin dns pool and you're done! Even that could be automated to respond dynamically to traffic.

Imagine the long-term cost-savings from only having to build your app and architecture once and not having to go through a more complex scaling process.

Certainly you should only worry about scaling after your app is up and running, but what if you didn't have to ever worry about it? That's the goal of this framework.

This is pre-alpha of course and I'm still in the development and experimentation stage, but I wanted to share with the community and get some feedback!

Update: To follow the feedback on this topic, see: http://news.ycombinator.com/comments?id=10001

Update2: I have decided to change my strategy a bit to bypass some of the limitations inherent in SQLite and S3DFS. Rather than have the user database files sit directly on S3 via S3DFS, I will have them reside on the EC2 instance and just back them up frequently to S3 via S3DFS. This will be faster, less complex, and more stable. When I get to the scaling part of the app, I'll probably use a reverse-proxy solution and divide the user database files across multiple EC2 instances. Thanks to the guys at Lunchgeeks for their help on this!

Share This

VC Rick Segal's approach seems the most charitable.

His tact works similarly in dating.
(Note that I date with the goal of marriage in mind - I'm definitely not a 'player', which makes this a good analogy to the good VC's - they're hoping to 'marry' founders when it's a good match)

Many guys burn bridges when they have to break up with a girl. However, that's totally unnecessary! I've certainly done that when I was a teen, but have found the charitable approach much more effective. Let me be clear that 'charitable' does not mean unclear or deceptive in the breakup. On the contrary - it's being very clear and honest about the breakup with the girl. That being said, you *do* withhold any information that would unnecessarily hurt her (ex: you're not attracted to something about her that she can't help).

The breakup needs to establish that:
1) the relationship is not a match for you, but that that is no reflection on her personally in the slightest
2) you genuinely admire the N number of amazing things about her
3) you hope that you both can still be good friends, but that you understand and honor her wish if she thinks it's better not to
4) the romantic possibilities of your relationship are just not there sufficient for marriage and never will be (let her move on with closure!) Don't worry, if you change your mind later you can crawl back on your knees to her and tell her you were wrong - but you have to let her go and pursue other possibilities.
5) you are open to talk with her about it at any time - this will usually relieve a lot of her anxiety about having to wonder why you broke up later.

Note: Be careful about the 'still being friends' thing. If you are hanging out too much with her, most other guys will avoid asking her out - and you don't want to hinder her ability to move on. Be sure to honor her space and let her know that you are doing that so she doesn't feel rejected by it.

Success stories:

2004 girlfriend: Good breakup, she decided it was better for her to not have contact. I honored that and imagine she has gone on to great things.

2005 girlfriend: Good breakup, though we did get back together several times. Ultimately it ended, but our communication was so good that we have remained best friends since then and talk regularly about our new relationships. We both are huge advocates of each other - she sells me to other girls and I sell her to other guys. We do business together too since there is such a high level of trust.

2006 girlfriend: Had a great time together, but we both knew it wouldn't work. We had great communication and though it was hard for me to let her go, I did and that really impressed her to the point where we have a similar high-trust, high-advocate relationship as with 2005 girlfriend.

2007 girlfriend: Ended pretty recently. So far so good. Very hard for both of us. But we've had good communication since and it is evolving well into a good friend relationship.

Note that these tips assume that you are only dating reasonably sane girls - with psycho chics this probably won't apply ;)

It's important to remember that going the charitable route is often harder in the short-term than the alternatives, but it's important to suck it up and have the kahoonas to do it right. Some guys may be tempted to not give her a clean break or feel they have to diminish her as a person to satisfy their own insecurities. DON'T DO IT! Seriously, it will hammer your karma and make life much more difficult for you in the long-term. To illustrate: 2006 girlfriend only dated me because of the good things 2005 girlfriend told her about how good our relationship and breakup was. The same with 2007 girlfriend. It's really the way to go - be respectful and honor them, because more often than not, they will return the sentiment.

Granted that my experience is pretty limited, please share your own success stories in the comments!

Update: When thinking more about this, I find that it applies also very well to leaving a job with the best possible feelings.

Share This

http://feeds.feedburner.com/Ycombinator-StartupSchool

This podcast contains all the published podcasts from the last 3 years of Startup School. Talks from 2005 and 2006 are linked to the official recordings and those from 2007 are linked to my lower quality recordings until the official ones are released.

If you were not there this year and are short on time - I'd recommend listening to Paul Buchheit, Paul Graham, and Greg McAdoo first.

Share This

(Download MP3)
I'm in Austin at the South by Southwest Interactive festival where Kathy Sierra gave the opening remarks (totally amazing as usual). Be sure to see her visuals at Brian Fitzgerald's blog - they are pretty important in understanding the talk.

Share This

Robert Sapolsky, the author of Why Zebras Don't Get Ulcers promotes the theory that irritable bowel syndrome (IBS) stems from our ancestors' need to quickly lose weight (by having a nice BM) when under high-stress (like running away from a pack of hyenas on the savanna). Similarly, many startups hinder their ability to run like the wind by having a 32 oz steak dinner right before the pack of hyenas comes to chase them. And people wonder why so many startups fail! Certainly there are capital-intensive startups that will need that steak - but those startups often don't eat the whole steak themselves but use it to trade for actual infrastructure (like a bicycle) and then get on it to race across the savanna.

We see a good example of the 'eat light' strategy from 37Signals:

Yeah. We've gotten quite a lot of VC calls. But one of the things we're seeing that we really don't care too much for is that way too many companies are taking money when they don't need it. And the whole idea we had was that having too little money is a great way of getting great product, because it's a way to get focused.

So we have definitely said to ourselves, "We don't want any outside money. We actually don't even want to grow our team." We're trying to design our products in a way that they can scale with more users without us having to scale as a company. So, through Signal vs. Noise, we're trying hard to deliver a pushback to companies that feel like they have to hire a bunch of people as early as possible and to take money to realize their vision by saying, "If your vision of your product costs a million bucks to make, try rescoping that idea in your head so it fits in $100K and get it out there earlier. Instead of having a 1-year product cycle, what could you do in 1 month?"

And sure, that doesn't work for every company, but in the web age, it works for way more companies than are trying to.

- From David Heinemeier Hansson's interview in Founders At Work: Stories of Startups' Early Days by Jessica Livingston

For extra bonus points, listen to Sapolsky's hilarious lecture on the subject (m4a quicktime file).

Share This