Archives for Adam Gotterer

PriceAdvance Day 7

Published on December 18, 2007

Priceadvance Logo

The last few days have been database/server optimization and scaling day! Before optimizing any of the default mySQL options we decided to test performance against a number of different database engines to see what would happen. As a side note mySQL wasn’t optimized from the beginning, we decided last minute our other box wasn’t going to handle the load well.

Lets try every reasonable mySQL engine…
Since our tables are read only, we tried packing the keys. This had no noticeable effect. With the packed keys we then compressed our core table. We managed to drop the table size from 1.4GB to 1.1GB, which is a pretty significant decrease. This also had no worthwhile effect. Next move was converting the tables over to innoDB. (note: you cant convert compressed tables to innoDB, so un-pack them first :P). My original theory was queries were slow because of table locking, so innoDB would offer us row level locking. That was a wrong assumption and innoDB caused a performance decrease by about 4 fold. The next and last option we tried was MEMORY (old Heap format). A 1.4GB table didn’t fit in the 1.6GB of allocated memory we gave to the engine. It got about 40% of the rows in and tapped out. We ran some queries on the 2.x million rows that actually made it in and the results were decently quick, but this wasn’t an optimal solution on our 2 gigs of RAM.

We finally gave in and decided to really fine tune and optimize mySQL and Apache. It took about 4 hours to get everything exactly how we wanted it. Right off the bat queries responded infinitely faster. Tomorrow will be judgment day, but I don’t expect to see anymore database issues until traffic increases again. Right now our traffic has stabilized and we have a nice core user base for our beta program. We will now begin phase two of operation save you lots of money, and do a full launch soon…

The lesson here is optimizing mySQL should be your number one configuration priority in database applications. Apache only needed a few tweaks, but the mySQL defaults are designed for small sites on low end servers. If the configuration ends up being successful I will post the file.

Filed in Media Next, Priceadvance, mySQL | 2 replies

Priceadvance Day 3

Published on December 13, 2007

Priceadvance Logo

I’m three days late to this post, but Roddy and I launched the beta of Priceadvance. Just five weeks after our Ycombinator interview (we didn’t get accepted). Priceadvance is a Firefox extension that finds better prices for products you are already viewing. Our objective is to cut out the process of doing price comparison research. Instead you should focus on finding products you want while we find the best prices.

The beta version is only a fraction of what we plan to accomplish with Priceadvance. We did a lot of internal testing and found our software to have a relative low number of bugs, now with over 3,000 installs we haven’t received a single bug report yet. The reason we released early was so we could determine if market for this type of product existed and if it did, what were the expectations. If 90% of the feedback we received was negative we would most likely be moving on to the next project.

When the idea for a Firefox extension first presented itself, we predicted the number one barrier to use would be the download/install. Our analytics show that around 40% of the visitors that went to Priceadvance.com installed the application.

One of the big unknown’s for us was what to expect as far as traffic. All we knew was we needed a dedicated box, mainly for customization reasons. We leased one from Layeredtech with decent specs (something like 1.8 Core Duo, 2 gigs of RAM and a good hard drive). In the first 24 hours after launch we landed on the front page of Lifehacker, Mashable and The Consumerist. Over the course of three days our traffic went from zero to over 12,000 page views. A Google search for “Priceadvance” on Monday returned 120 results, almost all of them unrelated to us. A search now returns over 3,000 results.

Working full time on CollegeHumor and Bustedtees has given me a substantial amount of experience in building applications that scale. Priceadvance is running a custom PHP framework, a “somewhat” optimized mySQL database using MyISAM tables and a Memcached cache layer. Even with my experience at CollegeHumor, we very rarely query tables with rows in the millions. The Priceadvance database has several million rows and appears to be our biggest bottleneck. Thankfully our server hasn’t been totally killed at any point, but slow response times are common. As I’m writing this, I’m killing slow queries.

We’re not positive what the answer to our database problem is. We are in the process of converting all our tables over to innoDB to test for the next day or two. Our queries are pretty simple and one theory is that the table level locking is what’s holding us back. If the innoDB experiment doesn’t work we may try Heap tables. If all of this fails, we will split the tables into several smaller tables and devise a mapping system for requests. We are also looking into adopting the Google server methodology and dumping our expensive machine in exchange for 2 or 3 lower end boxes for the same price. With all that said, I’m sort of glad we are having scaling issues only three days in.

So far this has been an incredible experience. Everything from the Ycombinator interview (I will write about that someday) to a friend telling me Priceadvance was the talk of his office. Take a look at the site; we appreciate all feedback, suggestions and comments.

Filed in Media Next, Priceadvance | One reply

Tags: , , ,

Firefox Extension – Preference Handeling

Published on December 6, 2007

I wrote a class to deal with preference handling in the PriceAdvance Firefox extension. I found the Mozilla documentation to be difficult to understand. So, I decided to share the class. Theres a little bug in Firefox that took me two days and borderline suicide to figure out… Extensions can have a default preference file that you can query using the preference interface. These preference files are “magically” loaded from a very specific file folder/location – “defaults\preferences\xxxxx.js”. To set a preference you simply put “pref(”branch_title.variable_name”, value);” in the file. If you use the class I made these preferences can be accessed like this: “Preference.getPref(”variable_name”)”. Now the bug I found is with the way you configure your extension. If the extension is bundled and installed “properly” it will work without a hitch. For development purposes I have my extension using a pointer from the extension folder to another folder location where I do all the work. Apparently when its setup as a pointer it wont load the preference files. Whats really annoying is theres no way to verify if the magical files have been loaded or not. Because of this I have avoided preference defaults and use my own variables class.

Preferences = {
initialize: function() {
  this.prefs = Components.classes['@mozilla.org/preferences-service;1']
  .getService(Components.interfaces.nsIPrefService)
  .getBranch('extensions.branch_title.');

  this.prefs.QueryInterface(Components.interfaces.nsIPrefBranch2);
},

getPref: function(name) {
  var type = this.prefs.getPrefType(name);

  if (type == this.prefs.PREF_STRING)
    return this.prefs.getCharPref(name);
  else if (type == this.prefs.PREF_INT)
    return this.prefs.getIntPref(name);
  else if (type == this.prefs.PREF_BOOL)
   return this.prefs.getBoolPref(name);
},

setPref: function(name, value) {
  var type = this.prefs.getPrefType(name);

  if(type == 0) {
    if(typeof(value) == 'number')
      type = 64;
    else if(typeof(value) == 'string')
      type = 32;
    else if(typeof(value) == 'boolean')
      type = 128;
  }

  if (type == this.prefs.PREF_STRING)
    this.prefs.setCharPref(name, value);
  else if (type == this.prefs.PREF_INT)
    this.prefs.setIntPref(name, value);
  else if (type == this.prefs.PREF_BOOL)
    this.prefs.setBoolPref(name, value);
}
};

// USAGE:
Preferences.initialize();
Preference.setPref('this_works', true);
alert(Preference.getPref('this_works'));
/******************/

To view preference variables that have been set, type “about:config” into the address bar and filter for your branch.

Filed in FireFox Extension, JavaScript | 2 replies

Old Things Still Rule!

Published on December 1, 2007

Earlier this week I was out to lunch with a College Humor colleague. He happens to be the only Connected Ventures employee with a kid. We got talking about toys. He told me his son plays with Ghostbusters and Thomas the train. Has the toy industry really not done any innovation in the last 20 years to capture kid’s attention?

I was on the subway heading uptown to meet a friend and I saw a billboard for this new TV series about the Wizard of oz. From the billboard I gathered it would be about the other side (not Pink Floyd) of Oz. Maybe it’s about what the tin man’s favorite drink is, or that the lion was nailing Dorothy on the side? Either way, it’s interesting that so many years later that’s what people are still into.

Year after year the top rated rock and roll songs are from the 80’s. The coolest cars are from the 40’s to 70’s. People collect just about anything that’s old, it obvious that is has some value to someone. What it comes down to, is a lot of the things we grew up with are still awesome today. My question is, in 20 years will we look back at things from this time and think the same way? Because there a very few things that I have recently discovered that are leaving a lasting impression. But I will never forget the first time I got a plasma screen TV, it must have been a similar experience to what it was like to get a color TV.

Filed in General | No replies

Whats going on?

Published on November 24, 2007

My apologies for not updating the last few days. I have been really busy with some stuff…

Roddy (my partner) and I just finalized and registered Media Next, Inc, which is now the parent company of PriceAdvance. We got this sweet embosser as well! PriceAdvance is the project we have been working on the past few months that is going to slap price comparison in the face! It was never intended to be a stealth startup, but it seems a lot of people started taking interest in what we were doing, so we decided to keep things under wraps until we launch. If things continue to go smoothly thats going to be really soon! One of the main purposes of this blog was to log my experience with building PriceAdvance from the ground up. After we launch I will backtrack a little and keep a real log of whats going on moving forward.

I also started hacking together the newest version of Prototype into the XUL library I have recently been maintaining. It was a little annoying that I released 1.6 RC1 only hours before they released 1.6 final. But I should have something up in the next day or two.

Filed in General | No replies

CollegeHumor is hiring!

Published on November 18, 2007

CollegeHumor Logo

 

CollegeHumor is looking to fill two development positions…

Flash / Action Script Developer

PHP /MySQL Developer

 

Connected Ventures is cool, but what could better then working with me?

Filed in CollegeHumor, Connected Ventures, Programming | No replies

Tags: , , , , ,

CollegeHumor 2008 Power Rankings

Published on November 13, 2007

CollegeHumor Logo

The CollegeHumor Power Rankings are an anual index of the top colleges in the U.S. for having the maximum amount of fun while putting forth the least amount of effort. Scoring is done on only the most competitive and important categories.

Percent Female
Average SAT
# pics uploaded to CH
Percent Greek
Bar Closing Time
Freshman Retention Rate
Percent Commuters
Stadium Capacity
Average Temperature in February
Closest Taco Bell
Billboard Peak
Cute College Girls on CH

Filed in CollegeHumor | No replies

Tags: ,

JavaScript Rich Text Editors

Published on November 11, 2007

For the past week I have been working on this custom JavaScript rich text editor for the CollegeHumor article writer. Our current editor is using FCKeditor, which has been decent but has only recently become compatible with all browsers. For the re-write we messed around with the ever so popular TinyMCE, which in my opinion I liked better then FCK. I found the problem with both of them to be customization and overhead. There’s a handful of other editors out there that we messed around with, but I found these two to be the most popular, have the best documentation and supporting community.

If you are going to use all of the features the full package offers, writing one from scratch is going to be a daunting task. Why re-invent the wheel? TinyMCE has a pretty sweet setup that lets you easily create themes and plugins. They even give you access to override some of the core functionality without touching the core files. In the CollegeHumor situation we are looking to customize almost every action and eliminate the overhead of the features we don’t plan to use. The CH article writer is clearly not one of the most visited pages on the site, but I cant imagine how it would perform if it was. These out of box editors are thousands of lines of JavaScript, tons of includes, graphics and html files. If you used something like this as the primary editor for say a large social site, it would be a waste of bandwidth and suffer from slow page loads on the client side.

So if you decide you want to code your own, Microsoft and Mozilla both have interfaces that ease creating custom rich text editors. For the basic features; bold, italic, justify, font size, etc. there is a single line commands that deal with the actual placement and HTML. Basically all it takes to get a super basic editor is an IFrame set to “design mode” and a few command buttons.

Microsoft execCommand docs
Mozilla execCommand docs
Browser Compatibility chart

If you end up dynamically generating the IFrame, beware of a quirk in Mozilla that requires the IFrame to be completely loaded before it can be set to design mode. The way I dealt with this was to put an event handler on window/load that triggers the design mode once I know everything has been downloaded. I have seen a few examples use short timers to set design mode after the IFrame was appended. I found this to work most of the time, but sometimes if the browser hiccuped an extra second it would try and add it before the IFrame was loaded, which threw an error.

Filed in JavaScript | 2 replies

24: The Unaired 1994 Piolot

Published on November 8, 2007

Since Fox will not be airing a new season of “24″ due to the writers’ strike, they plan to air a season that was originally filmed in 1994. CollegeHumor obtained this exclusive clip.

* There should be no excuse next year for “24″ sucking as much as it did this past season. If the strike ever ends, they will have an entire year to rewrite and re-shoot the season. I hope everyone is ready for a shit load of reality TV!

Filed in CollegeHumor, General | No replies

Tags: , ,

Help patrick find the girl of his dreams…

Published on November 7, 2007

Patrick was on the subway the other day and saw the girl of his dreams, but she got away. Heres his story…


Patrick discusses the girl of his dreams from Jakob Lodwick on Vimeo.

Detailed Drawing
NY girl of my dreams
(click image to enlarge)

If you have any information that could lead to Patrick finding his girl, please visit nygirlofmydreams.com.

Filed in General | No replies