I’ve been using git pretty heavily lately. The commands are a little wonky to learn, but once you’ve mastered a core set, distributed software revision control becomes a snap. I’m particularly impressed with how smoothly git manages branches and synchronization. I remember how painful that was in Subversion (and CVS).
Slideshare Down for the Count
Started to get all of the KeenerTech.com presentations online via SlideShare, but they seem to be down for the count. All major functionality (uploads, etc.) seems to be down except for, fortunately, support for embedded presentations. You’d think they’d have some sort of contingency plan in effect or some kind of staff notification feature.
JRuby Teams Jumps to EngineYard
Oracle is in the process of acquiring Sun, where three core team members of the JRuby development team work. Charles Nutter and company have jumped ship to join EngineYard, a prominent hosting company and a major supporter of the Ruby community. In fact, they’re one of the sponsors of our local RubyNation Conference. There’s more information about this on Slashdot.org.
Subversion Branching and Merging
This blog entry by Ned Batchelder is basically the best article I’ve seen on branching and merging using Subversion.
Note: (2018/01/13) Er, it’s a good article. But I’ve since moved on to git, a technology that makes branching and merging much, much easier. Friends don’t let friends do Subversion anymore.
Hosting with WebFaction
Based on recommendations from several of my fellow web professionals, including Naveen Vemula and Allan Bunch, I’ve chosen WebFaction.com for my Rails web hosting needs. I have 4 web sites that will be moved to Rails over the next few months, including KeenerTech.com.
Managing Database Dumps Using MySQL
When you’re developing applications that use a MySQL database, it’s often beneficial to work with real data. Since you never want to develop against your production database, it would be nice if you could take a snapshot of your production database, load it into a local database of your own and develop against the copy.
In fact, it’s not hard to do this at all.
Dumping a MySQL Database
Here’s the command to dump a MySQL database on a UNIX/Linux system:
mysqldump -h <host> -u <user> -p <database> | zip > staging.zip
When using the above command, it will prompt you for the password, which seems to work better than trying to pass the password as a parameter.
I typically also pipe the dump output to the zip command because the databases I usually deal with are generally large enough that it’s difficult to find enough room to store them on disk in an uncompressed form. Since a MySQL dump file is just text, it tends to compress by 75% or more.
Loading a Database Dump
Download the dump file from whatever machine you created it on. Unzip the file, which will create a local file called “-“. Now, this file name was created as an artifact of the original zip process, and it’s not a very friendly file name. So rename “-” to something more useful, like “dump.sql”.
In order to load the dump file, you’re going to have to take care of a few prerequisites, which should be fairly logical. You need to make sure that the database already exists, and is empty.
You also need to make sure that the user name you’re going to use when loading the dump file exists and has the privileges to create all of the necessary database objects. If necessary, you can use the following MySQL command to give a user all privileges:
GRANT ALL ON *.* TO someuser@somehost
To load the dump file, run the following command:
mysql -h <host> -u <user> –password=<pass> <database> < dump.sql
That’s all there is to it. You should now be able to easily dump and load a MySQL database in order to support your development efforts.
A Primer on RESTful Web Services
REST (Representational State Transfer) is a set of architectural principles defining how a system’s resources, and the states of those resources, can be managed over a network using only simple HTTP (Hyper-Text Transfer Protocol). The concepts behind REST were first introduced in 2000 by Roy Fielding at the University of California, Irvine. Nobody realized at the time how influential his dissertation, “Architectural Styles and the Design of Network-based Software Architectures,” would prove to be.
In retrospect, it’s not entirely surprising that the architectural principles Fielding presented in his paper were readily adopted. Fielding is the primary architect of the HTTP standard, a contributor to numerous other Internet standards and a founder of various open source tools, most notably the Apache HTTP Server. He has serious credentials, and if he claimed in his dissertation that there might be a better way to organize distributed web applications, then companies like Amazon, Yahoo and Google were prepared to listen.
Fielding’s dissertation came at a good time, because there was rampant dissatisfaction in the web services arena. There was certainly room for improvement. The whole web services strategy, which had seemed like a simple way to organize distributed systems, had become increasingly complex. Standards like SOAP and the Web Services Description Language (WSDL) didn’t really seem to help.
In many ways, REST was a deliberate return to the basics that made the World Wide Web itself successful.
The web arguably became successful because it was conceptually simple, yet powerful. In its purest form, there were web pages that had addresses (the ubiquitous URL), and users could view them using a web browser. A simple protocol like HTTP was all that web browsers needed in order to retrieve web pages. Furthermore, web pages supported links, which allowed users to traverse relationships between web pages.
With RESTful web services, the conceptual basis is resources, rather than web pages. A resource is anything that is important enough to be referenced as a distinct “thing.” For a company like Amazon, resources could be objects like products, publishers, customers, reviews, etc.
Since REST is really about accessing and manipulating resources, a RESTful approach for web services is also sometimes referred to as a resource-oriented architecture. Like web pages, each resource has a distinct URL from which information about the resource can be retrieved. Unlike web pages, it is possible to perform actions on resources, such as creating them, updating them, deleting them and viewing them. All of the actions that can be reasonably performed on a resource can be defined in a consistent manner using the HTTP standard.
HTTP Support for RESTful Web Services
The HTTP standard defines several different “methods” for performing standard operations on resources:
| GET | Retrieve a resource. |
| POST | Create a new resource. |
| PUT | Update a resource. |
| DELETE | Delete a resource. |
The GET and POST methods are already widely used by web browsers for retrieving web pages and for posting data from forms. REST goes one step further and stipulates that GET requests should be for viewing resources and should not cause any side effects, i.e. — a GET request should never cause a record to be created in a database. This was already considered to be a “good practice” in the web design field; adhering to this practice ensures that search engines and web crawlers won’t cause harmful side effects by traversing links on web pages.
The PUT and DELETE methods have been part of the HTTP standard for more than twelve years. However, they have not achieved widespread usage until recently, which is clearly a direct result of the spread of RESTful web services.
RESTful Actions
Think of resources as nouns, where each resource is represented by a clearly defined URL. The verbs are the HTTP methods (GET, POST, PUT and DELETE). The nouns and verbs can be combined into sentences that define actions to be performed on a resource.
| URL | HTTP Method | Action |
| http://example.com/books | GET | Get a list of books. |
| http://example.com/books | POST | Create a new book. |
| http://example.com/books/1 | GET | Show data for book 1. |
| http://example.com/books/1 | PUT | Update data for book 1. |
| http://example.com/books/1 | DELETE | Delete book 1. |
In the table above, the various URLs allow callers to perform actions on a book or, more specifically, the metadata associated with a book. The primary advantage of this architecture is that it offers a regular and predictable interface for managing most types of resources. For developers, using a RESTful web service is as simple as sending a request to a specific URL using the appropriate HTTP method. HTTP even supports the capability to send arguments with a request.
But there’s something even more important about how these requests are defined. The requests are stateless.
Each request is fully self-contained. All information needed for the processing of the request is included in the URL, the HTTP method, the HTTP headers or the content of the request. No knowledge of previous requests is required in order to perform an action.
The fact that the requests are stateless is vital for scalability. Run a load balancer in front of a fleet of computers, each running an instance of the web service. It doesn’t matter which computer requests get directed to because the requests are completely self-contained.
HTTP Status Codes and HTTP Headers
Each web service request sent via HTTP results in a response being sent back to the caller. At a minimum, the caller receives an HTTP status code that defines the status of the request. The HTTP standard defines numerous status codes that can easily be leveraged by a web service to provide meaningful information to a caller. A few of the most useful status codes are shown below:
| Code | Name | Description |
| 200 | OK | An action was successfully completed. |
| 201 | Created | A new resource was successfully created. |
| 400 | Bad Request | The server did not understand the request, which may have been malformed. |
| 401 | Not Authorized | The user was not authorized to perform the action. |
| 403 | Forbidden | The action was legal but the server will not complete the action. |
| 404 | Not Found | The specified resource was not found. |
| 405 | Method Not Allowed | The request was made using an HTTP method not supported by the resource. |
| 422 | Failed Validation | Originally defined by the WEBDAV standard. Indicates that the resource failed validation (typically on Create or Update). |
| 500 | Internal Server Error | A generic error that is returned by the server when something goes wrong. |
The status code generally provides enough information for a caller to determine whether a RESTful web service request succeeded or, if it failed, to gain some idea of why the request failed. In most cases, that’s all the caller needs.
The Create action is the exception. When a Create action is performed, the caller has no way to know the unique URL of the newly created resource. Accordingly, it has become commonly accepted practice in RESTful web services to return the URL of the new resource to the caller via the “Location” HTTP header. For consistency, some web services do this for both the Create and Update actions.
Representation Formats
Both the request and the response may optionally include content. For example, when creating a new book resource, the request will need to include information about the book. Most web services accept such content in easy-to-parse formats such as form-encoded parameters (the format for data that is POSTed from a traditional HTML form), JSON and XML. That data format is referred to as the representation format of the request.
Likewise, most web services respond to a request by sending back content in easy- to-parse formats such as JSON, XML or various flavors of XML such as Atom, RSS or XHTML. That data format is referred to as the representation format of the response. Callers can parse the content provided in the response and reuse it for their own purposes.
Many web services allow users to select from a choice of representation formats for the request and the response. For example, a caller might send a request that included JSON content and receive a response that included RSS content.
REST vs. SOAP
REST and SOAP represent two very different strategies for implementing web services. A RESTful architecture makes web services as conceptually simple as possible, with a standard and consistent approach for representing actions on resources. REST also makes no assumptions about the representation formats, such as XML or RSS, used for content that is sent back to callers.
SOAP is more elaborate than REST in many ways. There are numerous official standards layered on top of each other, providing features for remote procedure calls, security, etc. There is no common way of representing actions on resources. Instead, there is a standard approach for representing any type of content in XML in a generic fashion. With SOAP interfaces, the strategy is to have libraries that automatically parse complex SOAP-related XML and create objects based on the content, which can then be interrogated to retrieve data elements as needed.
In simplistic terms, REST focuses on providing a standard interface for resources, while SOAP focuses on providing a standard and generic representation for content. With REST, developers may have to create custom parse logic for content received from a RESTful web service. With SOAP, content can be parsed automatically and made available to the caller. In practice, REST representation formats tend to be much simpler than SOAP-related XML, so custom parsing is not typically a problem.
In general, RESTful web services are usually simpler and easier to use than SOAP interfaces. Conceptually, RESTful web services typically seem more object-oriented to end users, which enhances their perceived ease-of-use.
Additionally, SOAP interfaces often do not work well in cross-language environments, e.g. – SOAP clients in languages such as Perl, Ruby, Python and other languages may not work properly with SOAP web services. RESTful interfaces are usually less complex than SOAP interfaces, so language compatibilities are less of a problem.
Most importantly, implementing SOAP interfaces can be much more time- consuming than implementing RESTful interfaces, particularly in languages like Java.
References
The following references may be useful in understanding REST:
- Restful Web Services
2007 O’Reilly, by Leonard Richardson and Sam Ruby - “The Future of Web Services”
Online Video presentation by Gregg Pollack
http://www.railsenvy.com/2007/12/17/the-future-of-web-services - Roy Fielding Biography
http://roy.gbiv.com/ - Untangled: Roy Fielding’s Blog
http://roy.gbiv.com/untangled - HTTP Status Codes
http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
Introduction to RubyGems
RubyGems is a package manager for Ruby code. It’s used to organize reusable code into a deliverable known as a “gem.” Everyone who has programmed in Ruby or used the Rails framework has probably installed a gem at one time or another. Even the Rails framework is packaged as a set of gems.
The troubling thing is that many Ruby developers, and most Rails developers, are not taking full advantage of the power of gems. This article will explore some of the capabilities of RubyGems and illustrate how those capabilities can be used to support real-world Rails development efforts.
The Basics
RubyGems is the project which produced the well-known gem utility program. It shouldn’t surprise anyone that packages created by the utility are known as gems.
A gem has some key characteristics:
- It contains packaged code.
- It has a version associated with it.
- It may have dependencies on other gems.
The gem command can be used to install a gem:
$ gem install will_paginate
The example above installs the most recent version of the “will_paginate” gem.
$ gem install will_paginate --version 2.2.2
This example installs version 2.2.2 of the gem.
But where does RubyGems look to find the gem? The answer is that the gemcommand has a defined set of well-known locations where it will look for gems. This includes locations such as RubyForge, a free repository of open source Ruby code.
$ gem uninstall will_paginate
This command can be used to uninstall a gem. If there are multiple versions of the gem installed, then the command will prompt for the version to be removed.
$ gem update will_paginate
Update a gem to the latest version. To update all gems, just use “gem update.”
RubyGems “Admin” Features
You can do lot more with RubyGems than just install and uninstall gems. The gem command takes advantage of the information associated with each individual gem to make a variety of other features available to users.
$ gem list
This command will display a list of all the gems (including version numbers) that are installed locally. Note that you can have multiple versions of the same gem installed on a computer.
$ gem query --remote
This command will display a list of all the gems that are available remotely from the standard search locations.
$ gem spec will_paginate
Assuming the will_paginate gem is installed locally, this command will print out detailed information from the gem specification, including version, date created, summary, dependencies, the list of files included in the gem, etc.
$ gem environment
The gem command will also show you detailed information about the RubyGems environment. The above command produced the output in Listing 1.
Listing 1: RubyGems Environment
RubyGems Environment:
- RUBYGEMS VERSION: 1.1.0 (1.1.0)
- RUBY VERSION: 1.8.6 (2008-03-03
patchlevel 114) [i686-darwin9.0.0]
- INSTALLATION DIRECTORY: /usr/local/lib/ruby/gems/1.8
- RUBY EXECUTABLE: /usr/local/bin/ruby
- RUBYGEMS PLATFORMS:
- ruby
- x86-darwin-9
- GEM PATHS:
- /usr/local/lib/ruby/gems/1.8
- GEM CONFIGURATION:
- :update_sources => true
- :verbose => true
- :benchmark => false
- :backtrace => false
- :bulk_threshold => 1000
- :sources => ["http://gems.rubyforge.org",
"http://gems.rubyonrails.org"]
- REMOTE SOURCES:
- http://gems.rubyforge.org
- http://gems.rubyonrails.org
By viewing the output, developers can discover where the Ruby gems are installed on the local computer as well as other information. For example, the output also shows that the default locations where the gem command searches for gems are RubyForge and RubyOnRails.org.
RubyGems Documentation
There is a comprehensive online User Guide available for RubyGems. It can be found at the URL below:
I’d consider the RubyGems User Guide to be excellent if it was completely current. Since it is a little out-of-date in places, I’ll downgrade it slightly to good, rather than excellent. Despite a few discrepancies between the content and the current state of RubyGems, it’s still a useful manual.
Conclusion
RubyGems has been an extremely successful package. As such, it is now required by Rails 2.0.2 as a dependency. During the boot process, Rails also loads RubyGems, so there’s no longer a need for developers to use require statements to load RubyGems. This is not reflected in the online documentation, which indicates in many places the need to load RubyGems. Additionally, be aware that RubyGems options may have also changed since the documentation was written.
If developers are working with Ruby 1.8.6 (as most of us still are, at least for production code), RubyGems will have to be installed in each environment. Beginning with Ruby 1.9, RubyGems will be automatically delivered as part of a Ruby installation.
Rails Tips for Migrations
Migrations are a wonderful feature provided by the Rails framework. They provide an organized mechanism for building the database for an application, and are a distinct improvement over the random assortment of undocumented SQL scripts used by most projects.
The concepts behind migrations are so simple that you’d think that it would be difficult for anything to go awry.
You’d be wrong.
Here are some tips for getting the most effective use out of migrations in Rails.
- Down Methods: The down method of each migration should correctly back out the changes created by the up method. There is a tendency for developers to either skip the down method, or to implement the downmethod and then not test it effectively. Neither reason for an ineffective down method is acceptable.It is valid for some migrations to be non-reversible, but this should definitely not be the rule. Migrations that perform data conversions are often not reversible. Even if this is the case, a related group of migrations should be reversible whenever possible.
- Performance: Migrations are executed in development, test and production environments. To reduce the impact on customers, it is common for production applications to be brought down while database updates are occurring. A poorly performing migration can keep your production applications offline for an unacceptable amount of time. Remember that production databases often have much more data than development and test environments. Make sure that your migrations, particularly those that perform data conversions, run as efficiently as possible.
- DSL: Rails provides a nice Domain-Specific Language (DSL) for manipulating database tables in migrations. Use it.Only use SQL directly when the DSL cannot easily perform the task that is required. Using the DSL also has the added advantage of keeping your migrations database-agnostic, which will be beneficial if a change in database vendor is ever proclaimed by management.
- Sets: Sometimes new requirements are received that require a sweeping set of changes affecting multiple tables in a database. Often, the changes have no value individually, but must instead be implemented as a complete set in order to be useful. If the changes are relatively simple, they can all be incorporated into a single migration.On the other hand, more complex changes should probably be grouped into a set of migrations, with one migration for each table impacted. It’s up to teams to develop their own convention for organizing migrations into sets, since there is no generally accepted convention for Rails migrations. One possible solution is to use the second term of a migration name to indicate a set, if applicable. In the example below, “release2” is used to indicate that the migrations are part of a set.
046_release2_create_subscription.rb
047_release2_modify_distributor.rb
048_release2_modify_user.rb
049_release2_remove_old_tables.rb
The first term, of course, is either a sequential number or a timestamp, depending on how the development team wants to organize the migrations. Support for timestamps in migration names was added in Rails 2.0.
- Migration Sprawl: Sometimes the number of migrations mushrooms more than is needed. Imagine a situation in which numerous database changes are needed, and multiple people are working on the changes. Each person creates multiple migrations, some of them affecting the same tables. If all of the migrations are going to go into production at the same time, it may make sense for some consolidation to be done. For example, if there are six migrations affecting the same table, it may be more effective to have all of those changes done in one migration. The migrations will make more sense to users, and it will be easier for testers to verify that all of the changes have been done properly.
Dead Man Walking
This blistering article in the April issue of Fast Company exposes the management problems, lack of vision and business ineptitude that have transformed AOL from a high-flying Internet company and world-class brand-name into a slow-motion train wreck. The article is also strikingly negative about Randy Falco’s reign of error as the CEO of the organization.
Note: (2018/01/13) From my own experience at AOL, I think AOL is a technology company that never realized it was a technology company, with a tradition of hiring CEOs who never understood the Internet.
