Resumable Upload for the Google Data APIs
March 22, 2010
If you’re backing up local files to Google Docs using the Documents List API, there’s nothing more frustrating than having the upload fail moments before it completes. Unfortunately, current web protocols provide no guidance for reliably restarting failed uploads using HTTP. Because of this limitation, file uploads at Google and other sites have traditionally been limited to moderate sizes (e.g. 100 MB). For API services like the Google Documents List API and the YouTube API (both of which support large file uploads), this presents a major hurdle.
So today we’re rolling out a feature across the Google Data APIs that supports resumable POST/PUT HTTP requests in HTTP/1.0. The protocol was modeled after the ResumableHttpRequestsProposal suggested by Google Gears team. The protocol itself is straightforward. First your client makes an initial POST request to an API endpoint to obtain a unique upload URI:
Next, the file is sent in chunks to the server using the
For now, the resumable upload feature is only available in the DocList and YouTube APIs. The Java, Python, Objective-C, and .NET libraries have been updated to support the resumable functionality and documentation and client library samples can be found on code.google.com.
So today we’re rolling out a feature across the Google Data APIs that supports resumable POST/PUT HTTP requests in HTTP/1.0. The protocol was modeled after the ResumableHttpRequestsProposal suggested by Google Gears team. The protocol itself is straightforward. First your client makes an initial POST request to an API endpoint to obtain a unique upload URI:
POST /feeds/upload/create-session/default/private/full?convert=false HTTP/1.1
Host: docs.google.com
GData-Version: 3.0
Authorization: <your authorization header here>
Content-Length: 292
Content-Type: application/atom+xml
X-Upload-Content-Type: application/msword
X-Upload-Content-Length: 7654321
<?xml version='1.0' encoding='UTF-8'?>
<entry xmlns="http://www.w3.org/2005/Atom"
xmlns:docs="http://schemas.google.com/docs/2007">
<category scheme="http://schemas.google.com/g/2005#kind"
term="http://schemas.google.com/docs/2007#document"/>
<title>MyTitle</title>
<docs:writersCanInvite value="false"/>
</entry>
Next, the file is sent in chunks to the server using the
Content-Range header. This single request sends the first 100000 bytes of 1234567 byte PDF file:PUT <unique_upload_uri> HTTP/1.1
Host: docs.google.com
Content-Length: 100000
Content-Type: application/pdf
Content-Range: bytes 0-99999/1234567
<bytes 0-99999>
For now, the resumable upload feature is only available in the DocList and YouTube APIs. The Java, Python, Objective-C, and .NET libraries have been updated to support the resumable functionality and documentation and client library samples can be found on code.google.com.
