Guide for Using SIFT on Amazon Cloud Service

Many users have had trouble installing SIFT because it relies on old code from BLIMPS and BLAST. We have made a public image of SIFT available on the Amazon cloud so that you can run SIFT yourself, without having to bother with installation.

Setting up the Amazon EC2 Cloud Service
Setting up PuTTY
Running SIFT on AWS

Setting up the Amazon EC2 Cloud Service

Create a User Account

Amazon Web Services (AWS) can be accessed through here. To create an AWS account, click on "Create an AWS Account" at the top right hand corner of the page and follow the instructions. Please prepare your credit card information as it will be required for billing purposes.
Create_Account

You should see this page. Type in your email address and select "I am a new user".
Sign in Page

This will bring you to the registration page. You will need to fill in this page.

Upon clicking on the account, you will also need to fill in this page to create a Amazon Web Service (AWS) account.
Registration2

You will need to agree to the AWS Customer Agreement by checking the tick box. You will also need to type a series of letters and numbers for security check purpose.
Registration3

You will be brought to a page showing a price list for the different services.
Prices

At the bottom of the page, you will need to select a payment method.
Registration4

Next, Amazon will verify the creation of your account by calling you. You will need to provide a working number. Please note that US$1 will be charged for the call. When you click "Call me now", the webpage will show you a 4 digit code. You will also receive a call. It is an automated call. When prompted, key in the 4 digit code on the phone.
Verification

You will receive a confirmation through the automated voice machine as well as on the webpage.
Verification2

You can now sign into your new account with the email and password that you registered with.
Login2

You will again be reminded on the prices of the individual services.
Complete_Signup

Scroll down the page to verify your particulars and billing information. When you are done, click "complete sign up".
Billing Methods 2

Your subscription will be activated. You will be sent a confirmation through email.
Activating Subscription

You have completed the registration process.

Login to Cloud Service

Amazon Web Services (AWS) can be accessed through here. To sign in to AWS, you can either click on the "Sign in to the AWS Management console" on the top right hand corner or the "Sign in to the AWS console" which can also be found on the right side.
Login

To login to a particular service, you can modify the setting found on the right side below the orange-yellow button. Set it to Amazon EC2. (This is the service that we will be using for SIFT). Check the box that says "save this as your default console".

You will need to ensure that the Region is set to US East (Virginia). This is where the Amazon Machine Image (AMI) file for SIFT resides. You will not be able to find SIFT if the region is set to something else.

Setting up an instance

Once logged in, you will see a page similar to this. Select for EC2 tab and click the "Launch Instance" button.
Instance_Setup

Choose an AMI

A pop-up will appear, for you to choose the necessary Amazon Machine Image (AMI) file. Select Community AMIs.

At the option for viewing, select "Public Images" and search for SIFT. This will reduce the time needed to do the search. The result obtained will be similar as the figure shown below. The current ami you will select is ami-a0e51bc9. Click on "Select" button on the right to make a selection.

Instance Details

Once an AMI is selected, you will need to set the details for the instance. Number of instances and Availability zone can be left as default. Minimum selection of "Large" (m1.large) is needed in the Instance Type as the program is running at 64-bit. An ideal size will be "Extra Large" (m1.xlarge). Please note that the cost of usage varies with the different instance type. Once done, click continue to proceed.

In the advance instance options, you can choose the Kernel ID and Ram Disk ID to use with your instances. For Kernel ID, do not choose default . You will need to locate "aki-427d952b" as it is the kernel specified for SIFT. Default settings can be used for Ram Disk ID. It is a good practice to enable Termination Protection. This will ensure that instances will not be accidentally terminated. Once done, click continue to proceed.

Any value can be entered into the Key and Value field. This will be the title for your instance. Once done, click continue to proceed.

Creating Key Pairs

Key Pairs can be created here. This allows secure connection to your instance after it launches. There are few options for you to choose from. If you have created key pairs previously, you can select from the existing ones. If you haven't, you can create one or proceed without a key pair. It will be advisable to create one as the connection will be secure.

If you are creating a new key pair, remember to save your key pair file in a known location as it will be needed for authentication when accessed through SSH. This file has an extension of .pem. For example purpose, name it MyKeyPair.pem. Once done, click continue to proceed.

Configure Firewall

Next, you will need to edit the security settings. By default, Amazon blocks all ports. You will need to redefine the settings to enable yourself to access the instance you created. If you have existing security groups and you want to use them, you will need to make sure that the ports for SSH, SMTP, HTTP, HTTPS and MYSQL are opened. SIFT will need them to be open to work.

If you are creating a new security group, the ports you must define for SIFT are SSH, SMTP, HTTP, HTTPS and MYSQL. Please note that allowing more ports to be open makes your instances more accessible but at the same time less secure.

This is what you should see after adding the necessary ports.

Review & Launch

This page lists the AMI and all the settings that you have selected. Edit if there is any error. Click Launch when you are ready.

This page shows that your instances are in the process of launching. Click "View your instances on the Instances page" to view all the instances you have launched.

This is an example of the Instances page. Newly created instances will be running by default after launch.

Starting an instance

To start an instance that is not running, right click your instance and select start.

Status light will change from orange to yellow...

And finally green when it is up and running. To connect to the instance through SSH, the public DNS which is linked to your instance is needed. This can be found by right clicking on the instance and selecting "connect".

This is what you will see after clicking on connect. The address that is boxed up will be the temporary IP address for your instance as long as the instance is running. Once connected, you will now be charged according to the price plans.

Stopping an instance

Please note that only those that are running will be charged for their usage. As such, it will be very important to stop instances which are no longer needed. It is also very crucial to know the difference between terminate and stop. While initiating a "Stop" only "pauses the instance and free up resources", initiating a "Terminate" removed the whole instance from Amazon server. It is thus not recoverable.

If Termination Protection is enabled previously, clicking on Terminate will bring up this warning with no option of terminating the instance. (Act as a safety measure)

To Terminate a Termination Protected instance, we will need to click on "Change Termination Protection" to disable the protection feature. This is an additional security to ensure that you know and are sure of terminating the instance.

Setting up PuTTY

To login to AWS, you can use PuTTY. You will need to download PuTTY and PuTTYgen.

Installing PuTTY

PuTTY is a stand alone, executable file. PuTTY can be obtained from here.

Converting .pem file to .ppk file using PuTTYgen

You will need to download a program, PuTTYgen, for the conversion purpose. It can be found here.
Load MyKeyPair.pem file that is saved when generating Key Pair in AWS.

When the MyKeyPair.pem file is successfully loaded, a pop-up will appear.

Click ok. You have the option of whether to create a password or not. This password will be used for your login to AWS. Alternatively, if you did not wish to enter any password, you can still proceed. A prompt will ask if you want to save the key without a passphrase. Select Yes.

Do not click generate. Instead, select save private keys. The output file extension will be MyKeyPair.ppk

Connecting to AWS

Copy the address from AWS to PuTTY

At PuTTY, go to Connection > SSH > Auth
Click Browse to locate the MyKeyPair.ppk file that was previously created.

Once done, click open to proceed.

Running SIFT in AWS

Login to AWS

You will be shown a command window.

It will prompt you for a user id. Key in "ubuntu" If you had set a password earlier, you will need to key in your password. (The one that you generated along with the .ppk file)

You will then be shown the following screen:

This instance offers two options at the moment (Other features may be added upon request)
I. Running SIFT on any NCBI protein (SIFT Blink; Input: NCBI gi id)
II. Running SIFT on any protein sequence (SIFT Sequence; Input: fasta formatted-protein sequence)
Platform is Linux. Please refer to here (To be Updated) for the linux commands

Running SIFT on any NCBI protein

To run SIFT using pre-computed BLAST alignments (SIFT BLink), simply key in
cd sift4.0.4/bin
This will bring you to SIFT directory.

Format:
csh SIFT_for_submitting_NCBI_gi_id.csh <GI # from NCBI> <list of substitutions> <BEST or ALL>
Example:
csh SIFT_for_submitting_NCBI_gi_id.csh 22209009 ../test/gi22209009.subst BEST
Explanation:

Parameter	Example	Description
csh SIFT_for_submitting_NCBI_gi_id.csh		This is to call up the program that you will be using.
<GI # from NCBI>	22209009	This example uses http://www.ncbi.nlm.nih.gov/protein/22209009 22209009 (human CHK2 protein)
<list of substitutions>	../test/gi22209009.subst	This list the substitutions of interest
<BEST or ALL>	BEST	BEST or ALL. Specify whether to take the BEST hits from NCBI BLink, or ALL hits.

A whole lot of things will scroll across the screen, but ultimately you should see:

Results are in /home/ubuntu/sift4.0.4/tmp/22209009.SIFTprediction
To view file, use "more" command
more /home/ubuntu/sift4.0.4/tmp/22209009.SIFTprediction

For an explanation of the scores, please see SIFT help page.

Running SIFT on any protein sequence

You are logged into the AWS instance. To ensure you are in the correct folder, type:
cd sift4.0.4/bin or
cd /home/ubuntu/sift4.0.4/bin
Format:
csh SIFT_for_submitting_fasta_seq.csh <protein in fasta format> <database file> <substitution file>
Example:
csh SIFT_for_submitting_fasta_seq.csh ../test/lacI.fa /home/ubuntu/swissprot_db_01May2011/swissprot.fa ../test/lacI.subst
Explanation:

Parameter	Example	Description
csh SIFT_for_submitting_fasta_seq.csh		This is to call up the program that you will be using.
<protein in fasta format>	../test/lacI.fa	Protein sequence in fasta format
<database file>	/home/ubuntu/swissprot_db_01May2011/swissprot.fa	Database file
<substitution file>	../test/lacI.subst	Substitution file or "-" if none

**Please note that it is lacI . It the upper case of "i" and not lower case of L.
It will take a while to run.

To view results, type
more /home/ubuntu/sift4.0.4/tmp/lacI.SIFTprediction

Installing protein database (Optional)

Although SwissProt Database(May 1st, 2011) have been pre-installed as the default database, you can update the database locally by doing the following.

Setting up the Directory

Command	Description
cd /mnt	Change directory to mount
sudo chown ubuntu:ubuntu /mnt	Change ownership to user ubuntu
mkdir db	Make new directory db
cd db	Change directory to db

Types of Databases

Below list 3 databases and the instructions to obtain them.

Swiss-Prot(Recommended)
wget ftp://ftp.ebi.ac.uk/pub/databases/fastafiles/uniprot/uniprotkb_swissprot.gz

Once the file is downloaded, unzip the file by typing
zcat uniprotkb_swissprot.gz | awk '{if (/^>/) { print ">" $2} else { print $_}}' > swissprot.fa
The program is still running even though the screen seems to hang. It will take some time for the process to be completed.
Swiss-Prot/Trembl (long time for setup)
Follow directions for Swiss-Prot.
Type these codes instead:
wget ftp://ftp.ebi.ac.uk/pub/databases/fastafiles/uniprot/uniprotkb_trembl.gz
zcat uniprotkb_trembl.gz | awk '{if (/^>/) { print ">" $2 } else { print $_}}' > trembl.fa
To combine the Swiss-Prot and Trembl databases
cat trembl.fa swissprot.fa > swisstrembl.fa
NCBI nonredundant (long time for setup)
wget ftp://ftp.ncbi.nih.gov/blast/db/FASTA/nr.gz
zcat nr.gz | perl -pe 's/\|/ /g' | awk '{if (/^>/) { print $1 $2 " " $4} else {print $_}}' > nr.fa

Formatting the database you will be using.

SIFT code is *very picky* where special characters like "|" and too many characters in the protein name will cause an abort. As such, some extra formatting is needed and instructions to do so are provided below.

To format the file, type:
/home/ubuntu/sift4.0.4/blast-2.2.24/bin/formatdb -i /mnt/db/swissprot.fa -p T

Now you're ready to run SIFT!
Go to the executables:
cd /home/ubuntu/sift4.0.4/bin

Follow the steps as per running SIFT on any protein sequence. but replace the database file: </home/ubuntu/swissprot_db_01May2011/swissprot.fa> with </mnt/db/swissprot.fa> instead.
Command should look like this:
csh SIFT_for_submitting_fasta_seq.csh ../test/lacI.fa /mnt/db/swissprot.fa ../test/lacI.subst

To view the results:
more /home/ubuntu/sift4.0.4/tmp/lacI.SIFTprediction
more /home/ubuntu/sift4.0.4/tmp/lacI.alignedfasta

END OF GUIDE
Back to Top