The system was initially designed to cope with the levels of traffic and contributions associated with a large international news organsiation (thousands of requests per second / millions of content items).

The following approaches have been taken to ensure that the system is able to gracefully scale up to these targets.

Broadly speaking the Contribly API attempts to follow the approaches outlined in the 'Reactive Manifesto'.

API operations are generally asynchronous. For example when a media file is submitted, the application will immediately return HTTP 202 Accepted and a partial response rather than waiting for all of the required media processing operations to complete. This allows the client connection to be released as soon as possible. Priority is given to ensuring all incoming requests are captured.

Time consuming operations such as image processing and uploading of media files to remote locations are queued and processed in the background.

API responses are supplemented with additonal information as these background processes complete. Pushing as much processing as possible to the background leaves resources available to continue accepting new connections.

Under load the application should degrade gracefully, continuting to accept submissions but potentially taking longer for them to become fully processed.

The internal components of the system (persistance, search, image processing etc) all have the ability to scale out horizontally as required.

Service URLs provided to clients refer to CDNs or load balancers rather than single servers.

A typical use of the Contribly API might be to embed a gallery widget inside a page of an existing heavily trafficked website. This widget might use client side calls to the JSON API to render it's contents, generating multiple API calls from many unique clients. This widget would likely pull in thumbnails of the submissions, creating additional static asset requests from multiple clients.

User generated content applications such as galleries of user submitted images are typically read heavy. Connections to the JSON API are served via a CDN, allowing allowing API calls to be made directly from heavily trafficked websites.

A typical application serving a moderated collection of approved submissions should respond well to this caching approach with its read capacity limited by CDN rather than the API.

Public static assets are advertised on CDN URLs only (typically Amazon S3). The static file serving capacity of the system is therefore not limited by the API.

The core public facing write operation is the submission of new contributions.

A successful submission typically involves two API operations; the upload of a media file and a JSON POST of the contribution referencing the uploaded media file.

This endpoint simply has to receive the file and persist it. No additional media processing is done while the request is open.

The media file upload is likely to be large and long running. The processing of this request needs to be kept as simple as possible. The media is presented as raw bytes on the request body which can be streamed to persistant storage with minimal processing. The authentication (OAuth2) is independent of the content meaning the incoming file does not need to be read and processed in its entirety.

The contribution submission is a light weight JSON POST. Previous uploaded media files are referenced by previously assigned id so no binary data is involved in this request.

Referenced media files and contributions do not use incrementing sequences (which would need to be synchronised) so write operations can be scaled horizontially across multiple servers if needed.

A large media organisation uses the Contribly API and whitelabel website to host their user generated content microsite.

This microsite was launched in April 2013. This organisation also has integrated the Contribly API into Android and iOS mobile apps and called the API directly from Javascript interactives hosted in its main website.

Using only publicly available data it can be shown that in this time this service has attracted 220,000 approved contributions. The total number of contributions is likely to be higher on the assumpution that not every contribution is approved by moderators.

This installation typically attracts low three digit numbers of daily contributions but has had to response to several larger contribution surges (resulting from particular assignments 'going viral'). One such event resulted in 5,200 contributions within 48 hours.

The current production system has been tested into single digit millions of total contributions and double digit thousands of contributions per day range.