Searching NCBI has much more options than Google

(do you know Google options?)

By default the query text is searched in any part of any database

But you can specify the fields where you are looking for

• Title of a paper
• author
• date
• taxonomic id

protease NOT hiv1[organism]

This will limit a BLAST search to all proteases, except those in HIV 1.

1000:2000[slen]

This limits the search to entries with lengths between 1000 to 2000 bases for nucleotide entries, or 1000 to 2000 residues for protein entries.

Mus musculus[organism] AND biomol_mrna[properties]

This limits the search to mouse mRNA entries in the database. For common organisms, one can also select from the pulldown menu.

10000:100000[mlwt]

This is yet another example usage, which limits the search to protein sequences with calculated molecular weight between 10 kD to 100 kD.

src specimen voucher[properties]

This limits the search to entries that are annotated with a /specimen_voucher qualifier on the source feature.

all[filter] NOT environmental sample[filter] NOT metagenomes[orgn]

This excludes sequences from metagenome studies and uncultured sequences from anonymous environmental sample studies

That is a very good question

Each database has its own fields

Should this be a Homework?

Assembly

genome assemblies

Gene

genes from completely sequenced genomes and that have an active research community to contribute gene-specific data

Genome

sequence and map data from the whole genomes. The genomes represent both completely sequenced genomes and those with sequencing in-progress

EST

(Expressed Sequence Tag) sequences from GenBank. Typically short single-pass reads from cDNA libraries generated in survey projects

GSS

(Genome Survey Sequence) from GenBank. These are the genomic equivalent of EST records

Nucleotide

Apart from sequence data in the EST and GSS divisions, this database contains all the sequence data from GenBank

SRA

sequencing data from the next generation sequencing platforms

Taxonomy

names and phylogenetic lineages of the more than 350,000 species that have molecular data in the NCBI databases

MeSH

(Medical Subject Headings) controlled vocabulary and classification system (ontology) used for indexing articles in PubMed. MeSH terminology provides a consistent way to retrieve information that may use different terminology for the same concepts

Protein

amino acid sequences created from the translations of coding regions provided on nucleotide records in GenBank, also imported from the outside protein-only data sources (PIR, UniProtKB/Swiss-Prot, Protein Research Foundation, Protein Data Bank)

Protein Clusters

collection of related protein sequences (clusters) consisting of Reference Sequence proteins that are encoded by complete prokaryotic genomes as well those encoded eukaryotic organelle plasmids and genomes.

Conserved Domains

protein domains represented by sequence alignments and profiles for protein domains conserved in molecular evolution. It also includes alignments of the domains to known three-dimensional protein structures in the MMDB database.

Structure

Molecular Modeling Database (MMDB) contains experimental data from crystallographic and NMR structure determinations. The data for MMDB are obtained from the Protein Data Bank (PDB)

HomoloGene

automatically generated sets of homologous genes and their corresponding mRNA, genomic, and protein sequence data from selected eukaryotic organisms.

SNP

(Single Nucleotide Polymorphism) database is a central repository for single nucleotide polymorphisms, microsatellites, and small-scale insertions and deletions

BioProject

complete and incomplete (in-progress) large-scale molecular projects including genome sequencing and assembly, transcriptome, metagenomic, annotation, expression and mapping projects.

BioSample

contains descriptions of biological source materials used in studies that have data in other NCBI molecular databases such as Assembly, Nucleotide and SRA.

BioSystems

interacting sets of biomolecules involved in metabolic and signaling pathways, disease states, and other biological processes

Bookshelf

full-text books that can be searched online and that are linked to PubMed records

NCBI Web Site Search

database of static NCBI web pages, documentation, and online tools

NLM Catalog

records for books, journals, audiovisuals, computer software, electronic resources, and other materials in the National Library of Medicine (NLM) collections

dbGaP

(Database of Genotypes and Phenotypes) results of studies on the interaction of genotype and phenotype

dbVAR

(Database of Genomic Structural Variation) contains information about large-scale genomic variation, including large insertions, deletions, translocations and inversions

GEO Datasets

curated gene expression and molecular abundance data sets from the Gene Expression Omnibus (GEO) repository of microarray data

GEO Profiles

individual gene expression and molecular abundance profiles assembled from the Gene Expression Omnibus (GEO) repository

Probe

nucleic acid reagents designed for use in a wide variety of biomedical research applications including genotyping, gene expression studies, SNP discovery, genome mapping, and gene silencing

PubMed

database of citations and abstracts for biomedical literature from MEDLINE and additional life science journals

PubMed Central

(PMC) is the U.S. National Library of Medicine's digital archive of life sciences journal literature. PMC contains full-text manuscripts deposited by authors or articles provided by the publisher

• MedGen
• ClinVar
• OMIM
• PopSet
• PubChem BioAssay
• PubChem Compound
• PubChem Substance
• UniGene
• GTR

Entrez queries can be single words, short phrases, sentences, database identifiers, gene symbols, or names

AND: Finds documents that contain terms on both sides of the operator terms, the intersection of both searches.

OR: Finds documents that contain either term, the union of both searches.

NOT: Finds documents that contain the term on the left but not the term on the right of the operator, the subtraction of the right hand search from the one on the left

AND must be in uppercase. It is recommended to also use uppercase for OR and NOT

operators are processed left-to-right

promoters OR response elements NOT human AND mammals

Parenthesis can be used to control the evaluation order

g1p3 AND (response element OR promoter)

• Quotes " are important
• the fields are written inside brackets []

Each database page includes an Advanced Search option

• Certain fields can accept ranges of values

  • Publication Date, Modification Date, Accession, Molecular Weight, and Sequence Length.

• low and high numbers are entered with a colon “:” between them followed by the field

    110:500[Sequence Length]
    2015/3/1:2016/4/30[Publication Date]

The Clipboard is a temporary place on the NCBI website to save records.

• limited to 500 items on each database
• lost after eight hours of inactivity

My Collections that is a part of the My NCBI service is a more permanent place to save records.

You need to create an NCBI account to use My NCBI. It is easy and free

There are two major kinds of relationships in the Entrez system:

• computationally derived associations within a database (neighbors)
• relationships based on information present on the records themselves (hard links)

Combining neighbors and hard links can be an especially effective method for navigating across data and finding the most useful information

When we design molecular biology experiments, or when we analyze their results, we need to use several tools in chain

Today we are going to see an example using the NCBBI website

We need an official definition of each domain

F-box

• motif PF00646.
• alternative motifs (Gupta et al. 2015)
  • PF12937, PF13013, PF04300, PF07734, PF07735, PF08268 and PF08387

WD-40

• motif PF00400

Using NCBI Conserved Domain Architecture Retrieval Tool (https://www.ncbi.nlm.nih.gov/Structure/lexington/lexington.cgi)

Use the query:

[pfam00646,pfam00400]

to look for proteins that contain both domains in the specified order

• Several architectures contain both motifs
• Selected the first, with only the relevant domains and nothing else
• Click the button “Lookup sequences in Entrez” to find the list of proteins

Most of times is a good idea to check the Taxonomy database

Each sequence on GenBank is tagged with a taxon id

Using taxid is more precise than using common names

For example, a protein from human can be labeled “95% similar to mouse”

Is that a human or a mouse protein?

For your convenience you can download the sequences

• Decide Format
• Decide Content

In this case we only need accession ids

Now we use BLAST to find other proteins in legumes similar to the ones we have

Notice that CDART only has some proteins pre-processed. New sequences take time to be processed

How would you do that?

It is essential that your protocol can be replicated

It is a very good idea to save the search strategy in a file

It is also wise to save the output in a text file

Separate by tab or by comma

How many new proteins we find?

Do all of them have the good domains?

Let’s use CDD again, this time looking for motifs on the new proteins \[protein \to\{domains\}\] (the first time was \(domain\to\{proteins\}\))

• Keep only proteins with the good domains
• Download the sequences of all proteins
• Download the sequences of the messengers
• Design primers to measure gene expression
• Find literature about gene expression

And takes a lot of time

It is easy to make mistakes

It is hard to replicate

Can we do it automatically?

There are Entrez libraries for most languages

For example in R it is called rentrez

There is a command line version, and versions for all major computer languages

domains <- entrez_link(dbfrom = "protein", id=proteins,
                    by_id=TRUE, db="cdd")
doms <- lapply(domains,
            function(m) m$links$protein_cdd_concise_2)
udomains <- unique(unlist(doms))
udomains

 [1] "356469" "338561" "197608" "334189" "238121"
 [6] "225201" "351360" "337084" "294672" "312941"
[11] "197891"

dom.summary <- entrez_summary("cdd", udomains)
dom.title <- extract_from_esummary(dom.summary, "title")
prot.domains <- sapply(doms,
        function(d) paste(dom.title[d], collapse=" "))
unique(prot.domains)

 [1] "WD40 F-box-like"               
 [2] "WD40 FBOX"                     
 [3] "F-box-like WD40"               
 [4] "WD40 F-box"                    
 [5] "WD40 F-box-like WD40"          
 [6] ""                              
 [7] "WD40"                          
 [8] "WD40 WD40 F-box-like"          
 [9] "SGNH_hydrolase WD40 F-box-like"
[10] "WD40 LisH Dyp_perox"           
[11] "WD40 LisH"                     
[12] "WD40 Med15 LisH"               
[13] "WD40 LisH Med15"               
[14] "F-box"                         
[15] "WD40 Amelogenin LisH"          
[16] "WD40 WD40"                     

prot.seq <- entrez_fetch(db="protein", rettype = "fasta",
                                    id=proteins[has.both])
write(prot.seq, file="ncbi-proteins-wd40-fbox.faa")

messn <- entrez_link(dbfrom = "protein", id=proteins,
            by_id=TRUE, linkname="protein_nuccore_mrna")
messn <- sapply(messn, 
            function(m) m$links$protein_nuccore_mrna)
messn.seq <- entrez_fetch(db="nuccore", rettype = "fasta",
                          id=messn[has.both])
write(messn.seq, file="ncbi-messngr-wd40-fbox.fna")

genes <- entrez_link(dbfrom = "protein", by_id=TRUE,
         id=proteins[has.both], linkname="protein_gene")
ugenes <- sapply(genes, function(m) m$links$protein_gene)

geoprof <- entrez_link(dbfrom = "gene", id=ugenes,
            by_id=TRUE, linkname="gene_geoprofiles")
profiles <- lapply(geoprof,
            function(m) m$links$gene_geoprofiles)
ugeoprof <- unique(unlist(profiles))
geoprofiles_gds <- entrez_link(dbfrom = "geoprofiles",
              id=ugeoprof, linkname="geoprofiles_gds")

gene_pubmed <- entrez_link(dbfrom = "gene", id=ugenes,
            by_id=TRUE, linkname="gene_pubmed")
upubmed <- unique(unlist(lapply(gene_pubmed,
                    function(m) m$links$gene_pubmed)))
recs <- entrez_fetch(db="pubmed", id=upubmed,
                        rettype="xml")
papers <- parse_pubmed_xml(recs)

sonuc <- data.frame(proteins, 
            name=pnames[proteins], 
            messn=sapply(messn, 
                function(m) ifelse(is.null(m),"",m[1])), 
            domains=prot.domains,
            has.fbox,  has.wd40, has.both,
            stringsAsFactors = FALSE)