We are officially out of beta today! We launched with 19 supported merchants, an advanced pricing system that does in-cart pricing (the price after you add to cart) and an Internet Explorer version on top of a new Firefox release. Check out our new demo reel below, courtesy of mostlylisa.

The last few days have been database/server optimization and scaling day! Before optimizing any of the default mySQL options we decided to test performance against a number of different database engines to see what would happen. As a side note mySQL wasn’t optimized from the beginning, we decided last minute our other box wasn’t going to handle the load well.

Lets try every reasonable mySQL engine…
Since our tables are read only, we tried packing the keys. This had no noticeable effect. With the packed keys we then compressed our core table. We managed to drop the table size from 1.4GB to 1.1GB, which is a pretty significant decrease. This also had no worthwhile effect. Next move was converting the tables over to innoDB. (note: you cant convert compressed tables to innoDB, so un-pack them first :P). My original theory was queries were slow because of table locking, so innoDB would offer us row level locking. That was a wrong assumption and innoDB caused a performance decrease by about 4 fold. The next and last option we tried was MEMORY (old Heap format). A 1.4GB table didn’t fit in the 1.6GB of allocated memory we gave to the engine. It got about 40% of the rows in and tapped out. We ran some queries on the 2.x million rows that actually made it in and the results were decently quick, but this wasn’t an optimal solution on our 2 gigs of RAM.

We finally gave in and decided to really fine tune and optimize mySQL and Apache. It took about 4 hours to get everything exactly how we wanted it. Right off the bat queries responded infinitely faster. Tomorrow will be judgment day, but I don’t expect to see anymore database issues until traffic increases again. Right now our traffic has stabilized and we have a nice core user base for our beta program. We will now begin phase two of operation save you lots of money, and do a full launch soon…

The lesson here is optimizing mySQL should be your number one configuration priority in database applications. Apache only needed a few tweaks, but the mySQL defaults are designed for small sites on low end servers. If the configuration ends up being successful I will post the file.

I’m three days late to this post, but Roddy and I launched the beta of Priceadvance. Just five weeks after our Ycombinator interview (we didn’t get accepted). Priceadvance is a Firefox extension that finds better prices for products you are already viewing. Our objective is to cut out the process of doing price comparison research. Instead you should focus on finding products you want while we find the best prices.

The beta version is only a fraction of what we plan to accomplish with Priceadvance. We did a lot of internal testing and found our software to have a relative low number of bugs, now with over 3,000 installs we haven’t received a single bug report yet. The reason we released early was so we could determine if market for this type of product existed and if it did, what were the expectations. If 90% of the feedback we received was negative we would most likely be moving on to the next project.

When the idea for a Firefox extension first presented itself, we predicted the number one barrier to use would be the download/install. Our analytics show that around 40% of the visitors that went to Priceadvance.com installed the application.

One of the big unknown’s for us was what to expect as far as traffic. All we knew was we needed a dedicated box, mainly for customization reasons. We leased one from Layeredtech with decent specs (something like 1.8 Core Duo, 2 gigs of RAM and a good hard drive). In the first 24 hours after launch we landed on the front page of Lifehacker, Mashable and The Consumerist. Over the course of three days our traffic went from zero to over 12,000 page views. A Google search for “Priceadvance” on Monday returned 120 results, almost all of them unrelated to us. A search now returns over 3,000 results.

Working full time on CollegeHumor and Bustedtees has given me a substantial amount of experience in building applications that scale. Priceadvance is running a custom PHP framework, a “somewhat” optimized mySQL database using MyISAM tables and a Memcached cache layer. Even with my experience at CollegeHumor, we very rarely query tables with rows in the millions. The Priceadvance database has several million rows and appears to be our biggest bottleneck. Thankfully our server hasn’t been totally killed at any point, but slow response times are common. As I’m writing this, I’m killing slow queries.

We’re not positive what the answer to our database problem is. We are in the process of converting all our tables over to innoDB to test for the next day or two. Our queries are pretty simple and one theory is that the table level locking is what’s holding us back. If the innoDB experiment doesn’t work we may try Heap tables. If all of this fails, we will split the tables into several smaller tables and devise a mapping system for requests. We are also looking into adopting the Google server methodology and dumping our expensive machine in exchange for 2 or 3 lower end boxes for the same price. With all that said, I’m sort of glad we are having scaling issues only three days in.

So far this has been an incredible experience. Everything from the Ycombinator interview (I will write about that someday) to a friend telling me Priceadvance was the talk of his office. Take a look at the site; we appreciate all feedback, suggestions and comments.