Jose Carlos Monteiro is a Portuguese Ruby enthusiast and a mentor at RubyLearning.org. He has over 11 years of experience in programming with languages such as C, C++, Delphi, Java, Perl, PHP. He is also familiar with coding Unix Shell Scripts and used to working with version control systems (CVS and Subversion). His short-term goal is to acquire new skills, such as becoming more experienced in working with “modern” build tools (rake) and continuous integration tools. He also aims to become more familiar in applying modern concepts/methodologies such as TDD/BDD in his day-to-day work.

Introduction

Yahoo! Web Services are a set of online services that can be used by developers when creating “mashups”. These services are like a gateway to the huge amount of available data at Yahoo! The services are provided through a REST-based API, which is well documented at Yahoo! Developer Network.

Web Services

On Wikipedia one can read the following definition about Web Services: A Web Service is defined by the W3C as ‘a software system designed to support inter-operable Machine to Machine interaction over a network.’ Web services are frequently just Web APIs that can be accessed over a network, such as the Internet, and executed on a remote system hosting the requested services.

A Web Service is an online component that can be used by applications to receive or send data in a well-known XML format. Developers can use them to allow communication to happen between applications running in different operating systems and coded in distinct programming languages, since all communication is done in XML format.

RESTful Web Services

Again, on Wikipedia one can read the following definition to better understand the concept of RESTful Web Service: RESTful Web services attempt to emulate HTTP and similar protocols by constraining the interface to a set of well-known, standard operations (e.g., GET, PUT, DELETE). Here, the focus is on interacting with stateful resources, rather than messages or operations.

A more elaborate description is given on the article about RESTful Web Services, available through the Sun Developer Network: In the web services world, REpresentational State Transfer (REST) is a key design idiom that embraces a stateless client-server architecture in which the web services are viewed as resources and can be identified by their URLs. Web service clients that want to use these resources access a particular representation by transferring application content using a small globally defined set of remote methods that describe the action to be performed on the resource. The HTTP methods such as GET and POST are the verbs that the developer can use to describe the necessary create, read, update, and delete (CRUD) actions to be performed. Some may see an analogy to operations in SQL, which also relies on a few common verbs, however, the REST style and HTTP protocol are mutually exclusive, and REST does not require HTTP.

The term REST is an acronym for REpresentational State Transfer and was coined by Roy Fielding.

Mashups

One more, the definition of mashup on Wikipedia states that: A mashup is a web application that combines data from more than one source into a single integrated tool. Content used in mashups is typically sourced from a third party via a public interface or API (web services), although some in the community believe that cases where private interfaces are used should not count as mashups. Other methods of sourcing content for mashups include Web feeds (e.g. RSS or Atom), and screen scraping. Many people are experimenting with mashups using Amazon, eBay, Flickr, Google, Microsoft, Yahoo and YouTube APIs, which has led to the creation of the mashup editor.

Yahoo! Web Services and Ruby

Working with Yahoo! Web Services from Ruby is quite simple. One is only required to include the library Net::HTTP and use an Application ID that is obtained from Yahoo! Developer Network. For this how-to, Satish has already registered one such Application ID to be used in the code samples and in the following exercise(s).

Basically, one decides if the code is to use HTTP GET or HTTP POST methods and then sends a request and waits for a response. This response is in XML format and is well documented for each Yahoo! Web Service. The same goes for the request. By also including the REXML library one is able to handle the XML Response in a more readable way.

On Yahoo! Developer Network there are already pages with all that’s required to work with Yahoo! Web Services and parsing the XML Response. Also, Premshree Pillai has coded the Ruby API for Yahoo! Search Web Services that’s available inside the Search Developer Kit. The following code samples are taken almost straight from those pages, and don’t use the Ruby API for Yahoo! Search Web Services.

Code Samples

Using REST API with GET method:

<span style="color:blue">require 'net/http'

APPLICATION_ID = 'XIM4jgrV34GXdnMQfjYVcfY6YIMvb4FrOoBmcxAMHrVWrPPte0QDpYgieP5jZKFuCsL9'
YAHOO_WEB_SERVICE_SEARCH_URL = 'http://search.yahooapis.com/WebSearchService/V1/webSearch'
query = 'Satish Talim'
results_limit = 10
url = "#{YAHOO_WEB_SERVICE_SEARCH_URL}?appid=#{APPLICATION_ID}&query=#{URI.encode(query)}&results=#{results_limit}"
begin
  xml_result_set = Net::HTTP.get_response(URI.parse(url))
rescue Exception => e
  puts 'Connection error: ' + e.message
end
print xml_result_set.body
print xml_result_set.to_s + "\n"</span>

Explanation: The results_limit variable holds the maximum number of ResultSets in the returning response. That response is available on the xml_result_set variable, but to print it, one should call the body() method. The URL is constructed with the Yahoo! Web Service Search URL and with 3 arguments: appid, query and results. The query argument should be encoded to prevent non-HTML entities to be passed on the method Net::HTTP.get_response().

Using REST API with POST method:

<span style="color:blue">require 'net/http'

APPLICATION_ID = 'XIM4jgrV34GXdnMQfjYVcfY6YIMvb4FrOoBmcxAMHrVWrPPte0QDpYgieP5jZKFuCsL9'
YAHOO_WEB_SERVICE_SEARCH_URL = 'http://search.yahooapis.com/WebSearchService/V1/webSearch'
query = 'Satish Talim'
results_limit = 10
url  = YAHOO_WEB_SERVICE_SEARCH_URL

post_args     = {
  'appid'   => APPLICATION_ID,
  'query'   => query,
  'results' => results_limit,
}

begin
  response, xml_result_set = Net::HTTP.post_form(URI.parse(url), post_args)
rescue Exception => e
  puts 'Connection error: ' + e.message
end
puts xml_result_set
puts response</span>

Explanation: Same thing here as with GET, except that the method to use is Net::HTTP.post_form() which receives 2 arguments: the URL without any arguments and the arguments as a Hash.

Using REST API with GET method and parsing the XML Response with REXML:

<span style="color:blue">require 'net/http'
require 'rexml/document'

APPLICATION_ID = 'XIM4jgrV34GXdnMQfjYVcfY6YIMvb4FrOoBmcxAMHrVWrPPte0QDpYgieP5jZKFuCsL9'
YAHOO_WEB_SERVICE_SEARCH_URL = 'http://search.yahooapis.com/WebSearchService/V1/webSearch'
query         = 'Satish Talim'
results_limit = 10
url = "#{YAHOO_WEB_SERVICE_SEARCH_URL}?appid=#{APPLICATION_ID}&query=#{URI.encode(query)}&results=#{results_limit}"

begin
  xml_result_set = Net::HTTP.get_response(URI.parse(url))
rescue Exception => e
  puts 'Connection error: ' + e.message
end

result_set = REXML::Document.new(xml_result_set.body)
titles    = Array.new
summaries = Array.new
links     = Array.new

result_set.elements.each('ResultSet/Result/Title') do | element |
  titles << element.text
end
result_set.elements.each('ResultSet/Result/Summary') do | element |
  summaries << element.text
end
result_set.elements.each('ResultSet/Result/Url') do | element |
  links << element.text
end

titles.each_with_index do | title, index |
  puts "Title: #{title} - URL: #{links[index]}"
  puts "Summary: #{summaries[index]}"
  puts
end
</span>

Explanation: Exactly the same code up to the point of output. Now, instead of outputting the XML in raw format we use the class REXML::Document to parse the Result Nodes of the ResultSet into a set of Arrays: titles, summaries, links. This is just an example, since there are plenty more Response Fields that just these 3 and of course one can better control the query with other Request Parameters – as seen here: http://developer.yahoo.com/search/web/V1/webSearch.html.

Exercise

Code a class that accesses Yahoo! Web Service Utility getTime – http://developer.yahoo.com/util/timeservice/V1/getTime.html – which has methods that get the current time in Unix format and in milliseconds. It should have Exception Handling – when the ApplicationID is invalid or the URL isn’t properly parsed or the HTTP response is not OK.

Have fun!

Further Reading

List of Yahoo! Web Services

• http://www.yahooapis.com/util/ – Yahoo! Utility Web Services: The Utility Web Services are little helpers for applications that consume web services.
• http://developer.yahoo.com/openid/ – Yahoo! OpenID: OpenID makes it easy for you to deliver a simplified login experience for your users. Rather than having to create a unique user name and password for your website, users can now use an existing user name and password that they have created elsewhere. Of course, this is true only if “elsewhere” is a participating supplier of OpenIDs.
• http://developer.yahoo.com/mail/ – Yahoo! Mail Web Services: With the Yahoo! Mail Web Service APIs, you can build applications to perform tasks such as listing messages, displaying folders, and composing and sending messages.
• http://www.yahooapis.com/answers/ – Yahoo! Answers Web Services: Tap into the collective knowledge of millions of Yahoo! users with the Yahoo! Answers Web Services APIs. Search for expert advice on any topic, from within your very own site. Watch for new questions in the Answers categories of your choice. Keep track of fresh content from your favorite Answers experts, and more!
• http://developer.yahoo.com/search/ – Yahoo! Search Web Services: Yahoo! Search Web Services allow you to access Yahoo content and services in your favorite programming languages. This means you can now build Yahoo directly into your own applications. http://www.ysearchblog.com/archives/000084.html – A look inside the world of search from the people at Yahoo!
• http://developer.yahoo.com/local/ – Yahoo! Local Web Services: The content in Yahoo! Local makes a great addition to any mashup, bringing in location-based relevancy and the additional context of what real people have experienced in these places.
• http://developer.yahoo.com/maps/ – Yahoo! Maps Web Services: The Yahoo! Maps APIs allow you to embed rich and interactive maps into your web and desktop applications using the Flash, Ajax and Map Image APIs.
• http://developer.yahoo.com/travel/ – Yahoo! Travel Web Services: Yahoo! Travel enables you to plan your trips, explore destinations, find airfares, get ratings and reviews for hotels, find things to do, and share your plans with others. The Yahoo! Travel web services enable you to build applications that use the trip plan data from the Yahoo! Travel Trip Planner.
• http://www.yahooapis.com/shopping/ – Yahoo! Shopping Web Services: You can use Yahoo! Shopping Web Services to search their database of millions of product offers and thousands of merchants available through the Yahoo! Shopping site. The API can be used to create applications that provide comparison shopping and product search functionality as well as display user reviews.

List of available RubyForgeGems for Yahoo! Web Services

• http://rubyforge.org/projects/yahoo/ – A Ruby client for Yahoo Web Services.
• http://rubyforge.org/projects/activeyahoo/ – ActiveYahoo is a Ruby API for Yahoo! Web Services with a focus on encapsulating Yahoo’s responses into their own objects. (e.g., a WebSearch object will return a WebSearchResult object).
• http://rubyforge.org/projects/ysm4r/ – A Ruby client for Yahoo’s Search Marketing API, enabling Ruby coders to use their favorite language to access Yahoo’s Search Marketing services!
• http://rubyforge.org/projects/yahoolocal/ – Interface with Yahoo Local web service v2. Its Painfully easy: Step 1. Set your api key: Local.api_key = ‘yourapikeyhere’ Step 2. Search for late night munchies response = Local.search(‘pizza’, {:zip => 32304}).
• http://rubyforge.org/projects/unifiedsearch/ – Unified::Search is a Ruby library to provide a unified access wrapper to a number of different search webservices, currently supported are Google, Yahoo, MSN and Teoma. This Project Has Not Released Any Files.
• http://rubyforge.org/projects/graticule/ – Graticule is a geocoding API that provides a common interface to all the popular services (Google, Yahoo, Geocoder.us, and MetaCarta), distance calculations, and a command line interface.
• http://rubyforge.org/projects/upcoming/ – A Ruby wrapper for the Upcoming.org REST based API Version 1.0. Upcoming.org is a community website for discovering and sharing events. This Project Has Not Released Any Files.
• http://rubyforge.org/projects/upcoming-rb/ – A simple wrapper around the Yahoo! Upcoming API. This Project Has Not Released Any Files.

Thank you Jose for talking to the users of RubyLearning. In case the readers have any queries, questions on this article, kindly post your questions here and Jose would be glad to answer.

• Note: The information provided above is derived from the Yahoo! Developer Network and all Yahoo! based trademarks and logos are trademarks or registered trademarks of Yahoo.com in the United States and other countries.
• The above Ruby programs can be downloaded and tried out.