Title: | Single Cell Analysis and Differential Expression |
---|---|
Description: | Analyzing and interactively exploring large-scale single-cell RNA-seq datasets. 'pagoda2' primarily performs normalization and differential gene expression analysis, with an interactive application for exploring single-cell RNA-seq datasets. It performs basic tasks such as cell size normalization, gene variance normalization, and can be used to identify subpopulations and run differential expression within individual samples. 'pagoda2' was written to rapidly process modern large-scale scRNAseq datasets of approximately 1e6 cells. The companion web application allows users to explore which gene expression patterns form the different subpopulations within your data. The package also serves as the primary method for preprocessing data for conos, <https://github.com/kharchenkolab/conos>. This package interacts with data available through the 'p2data' package, which is available in a 'drat' repository. To access this data package, see the instructions at <https://github.com/kharchenkolab/pagoda2>. The size of the 'p2data' package is approximately 6 MB. |
Authors: | Nikolas Barkas [aut], Viktor Petukhov [aut], Peter Kharchenko [aut], Simon Steiger [ctb], Rasmus Rydbirk [ctb], Evan Biederstedt [cre, aut] |
Maintainer: | Evan Biederstedt <[email protected]> |
License: | GPL-3 |
Version: | 1.0.12 |
Built: | 2024-10-24 04:46:29 UTC |
Source: | https://github.com/kharchenkolab/pagoda2 |
armaCor - matrix column correlations. Allows faster matrix correlations with armadillo. Similar to cor() call, will calculate correlation between matrix columns
armaCor(mat)
armaCor(mat)
mat |
matrix |
matrix with columns as correlations
Perform basic 'pagoda2' processing, i.e. adjust variance, calculate pca reduction, make knn graph, identify clusters with multilevel, and generate largeVis and tSNE embeddings.
basicP2proc( cd, n.cores = 1, n.odgenes = 3000, nPcs = 100, k = 30, perplexity = 50, log.scale = TRUE, trim = 10, keep.genes = NULL, min.cells.per.gene = 0, min.transcripts.per.cell = 100, get.largevis = TRUE, get.tsne = TRUE, make.geneknn = TRUE )
basicP2proc( cd, n.cores = 1, n.odgenes = 3000, nPcs = 100, k = 30, perplexity = 50, log.scale = TRUE, trim = 10, keep.genes = NULL, min.cells.per.gene = 0, min.transcripts.per.cell = 100, get.largevis = TRUE, get.tsne = TRUE, make.geneknn = TRUE )
cd |
count matrix whereby rows are genes, columns are cells. |
n.cores |
numeric Number of cores to use (default=1) |
n.odgenes |
numeric Number of top overdispersed genes to use (dfault=3e3) |
nPcs |
numeric Number of PCs to use (default=100) |
k |
numeric Default number of neighbors to use in kNN graph (default=30) |
perplexity |
numeric Perplexity to use in generating tSNE and largeVis embeddings (default=50) |
log.scale |
boolean Whether to use log scale normalization (default=TRUE) |
trim |
numeric Number of cells to trim in winsorization (default=10) |
keep.genes |
optional set of genes to keep from being filtered out (even at low counts) (default=NULL) |
min.cells.per.gene |
numeric Minimal number of cells required for gene to be kept (unless listed in keep.genes) (default=0) |
min.transcripts.per.cell |
numeric Minimumal number of molecules/reads for a cell to be admitted (default=100) |
get.largevis |
boolean Whether to caluclate largeVis embedding (default=TRUE) |
get.tsne |
boolean Whether to calculate tSNE embedding (default=TRUE) |
make.geneknn |
boolean Whether pre-calculate gene kNN (for gene search) (default=TRUE) |
a new 'Pagoda2' object
Generate a 'pagoda2' web application from a 'Pagoda2' object
basicP2web(p2, app.title = "Pagoda2", extraWebMetadata = NULL, n.cores = 4)
basicP2web(p2, app.title = "Pagoda2", extraWebMetadata = NULL, n.cores = 4)
p2 |
a 'Pagoda2' object |
app.title |
name of application as displayed in the browser title (default='Pagoda2') |
extraWebMetadata |
additional metadata generated by p2.metadata.from.fractor (default=NULL) |
n.cores |
numeric Number of cores to use for differential expression calculation (default=4) |
a 'pagoda2' web object
Rescale the weights in an edge matrix to match a given perplexity. From 'largeVis', <https://github.com/elbamos/largeVis>
buildWijMatrix(x, threads = NULL, perplexity = 50)
buildWijMatrix(x, threads = NULL, perplexity = 50)
x |
An edgematrix, either an 'edgematrix' object or a sparse matrix. |
threads |
numeric The maximum number of threads to spawn (default=NULL). Determined automatically if NULL (default=NULL) |
perplexity |
numeric Given perplexity (default=50) |
A list
with the following components:
An [N,K] matrix of the distances to the nearest neighbors.
An [N,K] matrix of the node indexes of the neartest neighbors. Note that this matrix is 1-indexed, unlike most other matrices in this package.
The number of nearest neighbors.
Returns a list vector with the number of cells that are present in more than one selections in the provided p2 selection object
calcMulticlassified(sel)
calcMulticlassified(sel)
sel |
a pagoda2 selection as genereated by readPagoda2SelectionFile |
list vector with the number of cells that are present in more than one selections in the provided p2 selection object
Get the number of cells in each selection group
cellsPerSelectionGroup(selection)
cellsPerSelectionGroup(selection)
selection |
a pagoda2 selection list |
a named vector of cell numbers in each groups
Collapse aspect patterns into clusters
collapse.aspect.clusters(d, dw, ct, scale = TRUE, pick.top = FALSE)
collapse.aspect.clusters(d, dw, ct, scale = TRUE, pick.top = FALSE)
d |
matrix of normalized aspect patterns (rows: significant aspects, columns: cells), normally the output $xv in 'tamr', the combined pathways that show similar expression patterns |
dw |
corresponding weight matrix to parameter 'd' |
ct |
clusters, the output of fastcluster::hclust() |
scale |
boolean Whether to scale aspects (default=TRUE) |
pick.top |
boolean Whether to pick top aspects (default=FALSE) |
list of clusters from matrix of normalized aspect patterns and clusters from the corresponding weight matrix
Compare two different clusterings provided as factors by plotting a normalised heatmap
compareClusterings(cl1, cl2, filename = NA)
compareClusterings(cl1, cl2, filename = NA)
cl1 |
clustering 1, a named factor |
cl2 |
clustering 2, a named factor |
filename |
an optional filename to save the plot instead of displaying it, will be passed to pheatmap (default=NA) |
invisible summary table that gets plotted
Perform extended 'Pagoda2' processing. Generate organism specific GO environment and calculate pathway overdispersion.
extendedP2proc(p2, organism = "hs")
extendedP2proc(p2, organism = "hs")
p2 |
the 'Pagoda2' object |
organism |
character Organisms hs (Homo Sapiens), mm (M. Musculus, mouse) or dr (D. Rerio, zebrafish) (default='hs') |
list of a 'Pagoda2' object and go.env
Returns a factor of cell membership from a p2 selection object the factor only includes cells present in the selection. If the selection contains multiclassified cells an error is raised
factorFromP2Selection(sel, use.internal.name = FALSE, flatten = FALSE)
factorFromP2Selection(sel, use.internal.name = FALSE, flatten = FALSE)
sel |
a pagoda2 selection as genereated by readPagoda2SelectionFile |
use.internal.name |
boolean Whether to use field 'internal.name' as factor names (default=FALSE) |
flatten |
boolean Whether to ignore multiclassified cells, overwriting randomly (default=FALSE) |
factor of cell membership from a p2 selection object. The factor only includes cells present in the selection.
Converts a list of factors into 'pagoda2' metadata optionally filtering down to the cells present in the provided 'pagoda2' app.
factorListToMetadata(factor.list, p2 = NULL)
factorListToMetadata(factor.list, p2 = NULL)
factor.list |
list of factors named by the cell identifier |
p2 |
'pagoda2' app to filter the factors by, optional (default=NULL) |
'pagoda2' web metadata object
Converts a names factor to a p2 selection object if colors are provided it assigns those, otherwise uses a rainbow palette
factorToP2selection(cl, col = NULL)
factorToP2selection(cl, col = NULL)
cl |
factor |
col |
names vector of colors (default=NULL) |
a p2 selection object (list)
Filter cells based on gene/molecule dependency
gene.vs.molecule.cell.filter( countMatrix, min.cell.size = 500, max.cell.size = 50000, p.level = min(0.001, 1/ncol(countMatrix)), alpha = 0.1, plot = TRUE, do.par = TRUE )
gene.vs.molecule.cell.filter( countMatrix, min.cell.size = 500, max.cell.size = 50000, p.level = min(0.001, 1/ncol(countMatrix)), alpha = 0.1, plot = TRUE, do.par = TRUE )
countMatrix |
input count matrix to be filtered |
min.cell.size |
numeric Min allowed cell size (default=500) |
max.cell.size |
numeric Max allowed cell size (default=5e4) |
p.level |
numeric Statistical confidence level for deviation from the main trend, used for cell filtering (default=min(1e-3,1/ncol(countMatrix))) |
alpha |
numeric Shading of the confidence band (default=0.1) |
plot |
boolean Plot the molecule distribution and the gene/molecule dependency fit (default=TRUE) |
do.par |
boolean Reset graphical parameters prior to plotting (default=TRUE) |
a filtered matrix
Given a cell clustering (partitioning) and a set of user provided selections generate a cleaned up annotation of cluster groups that can be used for classification
generateClassificationAnnotation(clustering, selections)
generateClassificationAnnotation(clustering, selections)
clustering |
a factor that provides the clustering |
selections |
a p2 selection object that provided by the web interfact user |
a named factor that can be used for classification
Get a control geneset for cell scoring using the method described in Puram, Bernstein (Cell, 2018)
get.control.geneset(data, signature, n.bins = 25, n.genes.per.bin = 100)
get.control.geneset(data, signature, n.bins = 25, n.genes.per.bin = 100)
data |
matrix of expression, rows are cell, columns are genes |
signature |
character vector The signature to evaluate, a character vector of genes |
n.bins |
numeric Number of bins to put the genes in (default=25) |
n.genes.per.bin |
numeric Number of genes to get from each bin (default=100) |
a character vector that can be used as a background signature
Generate differential expression genesets for the web app given a cell grouping by calculating DE sets between each cell set and everything else
get.de.geneset(pagObj, groups, prefix = "de_")
get.de.geneset(pagObj, groups, prefix = "de_")
pagObj |
pagoda object |
groups |
named factor to do the de by |
prefix |
chararcter Prefix to assign to genesets generated (default="de_") |
a 'pagoda2' web object
Returns all the cells that are in the designated selections. Given a pagoda2 selections object and the names of some selections in it returns the names of the cells that are in these selections removed any duplicates
getCellsInSelections(p2selections, selectionNames)
getCellsInSelections(p2selections, selectionNames)
p2selections |
a p2 selections object |
selectionNames |
the names of some selections in th p2 object |
a character vector of cell names
Assign names to the clusters, given a clustering vector and a set of selections. This function will use a set of pagoda2 cell seletcion to identify the clusters in a a named factor. It is meant to be used to import user defined annotations that are defined as selections into a more formal categorization of cells that are defined by cluster. To help with this the function allows a percent of cells to have been classified in the selections into multiple groups, something which may be the result of the users making wrong selections. The percent of cells allows to be multiselected in any given group is defined by multiClassCutoff. Furthermore the method will assign each cluster to a selection only if the most popular cluster to the next most popular exceed the ambiguous.ratio in terms of cell numbers. If a cluster does not satisfy this condtiion it is not assigned.
getClusterLabelsFromSelection( clustering, selections, multiClassCutoff = 0.3, ambiguous.ratio = 0.5 )
getClusterLabelsFromSelection( clustering, selections, multiClassCutoff = 0.3, ambiguous.ratio = 0.5 )
clustering |
a named factor of clusters, where every entry is a cell |
selections |
a pagoda2 selection object |
multiClassCutoff |
numeric Percent of cells in any one cluster that can be multiassigned (default=0.3) |
ambiguous.ratio |
numeric Ratio of first and second cell numbers for any cluster to produce a valid clustering (default=0.5) |
a data.frame with two columns, one for cluster and one for selections, each cluster appears only once
Retrieves the colors of each selection from a p2 selection object as a names vector of strings
getColorsFromP2Selection(sel)
getColorsFromP2Selection(sel)
sel |
pagoda2 selection object |
a named vector of hex colours
Get a mapping form internal to external names for the specified selection object
getIntExtNamesP2Selection(x)
getIntExtNamesP2Selection(x)
x |
p2 selection object |
list of names from the specified selection object
Converts the output of hierarchical differential expression aspects into genesets that can be loaded into a 'pagoda2' web app to retrive the genes that make the geneset interactively
hierDiffToGenesets(output)
hierDiffToGenesets(output)
output |
output of getHierarchicalDiffExpressionAspects |
a geneset that can be loaded into p2 web genesets
Generate a Rook Server app from a 'Pagoda2' object. This generates a 'pagoda2' web object from a 'Pagoda2' object by automating steps that most users will want to run. This function is a wrapper about the 'pagoda2' web constructor. (Advanced users may wish to use that constructor directly.)
make.p2.app( r, dendrogramCellGroups, additionalMetadata = list(), geneSets, show.depth = TRUE, show.batch = TRUE, show.clusters = TRUE, appname = "Pagoda2 Application", innerOrder = NULL, orderDend = FALSE, appmetadata = NULL )
make.p2.app( r, dendrogramCellGroups, additionalMetadata = list(), geneSets, show.depth = TRUE, show.batch = TRUE, show.clusters = TRUE, appname = "Pagoda2 Application", innerOrder = NULL, orderDend = FALSE, appmetadata = NULL )
r |
a 'Pagoda2' object |
dendrogramCellGroups |
a named factor of cell groups, used to generate the main dendrogram, limits zoom in |
additionalMetadata |
a list of metadata other than depth, batch and cluster that are automatically added (default=list()) |
geneSets |
a list of genesets to show |
show.depth |
boolean Include depth as a metadata row (default=TRUE) |
show.batch |
boolean Include batch as a metadata row (default=TRUE) |
show.clusters |
boolean Include clusters as a metadata row (default=TRUE) |
appname |
character Application name (default="Pagoda2 Application") |
innerOrder |
Ordering of cells inside the clusters provided in dendrogramCellGroups (default=NULL). This should be one of "odPCA", "reductdist", "graphbased", "knn". Defaults to NULL |
orderDend |
boolean Whether to order dendrogram (default=FALSE) |
appmetadata |
a 'pagoda2' web application metadata (default=NULL) |
a 'pagoda2' web object that presents a Rook compatible interface
Scale the designated values between the range of 0 and 1
minMaxScale(x)
minMaxScale(x)
x |
values to scale |
the scaled values
example_matrix = matrix(rep(c(1:5), 3), 5) minMaxScale(example_matrix)
example_matrix = matrix(rep(c(1:5), 3), 5) minMaxScale(example_matrix)
Get a vector of the names of an object named by the names themselves. This is useful with lapply when passing names of objects as it ensures that the output list is also named.
namedNames(g)
namedNames(g)
g |
an objects on which we can call names() |
vector with names of object
Generate a GO environment for human for overdispersion analysis for the the back end
p2.generate.dr.go(r)
p2.generate.dr.go(r)
r |
a 'Pagoda2' object |
a GO environment object
Generate a GO environment for the organism specified
p2.generate.go( r, organism = NULL, go2all.egs = NULL, eg.alias2eg = NULL, min.env.length = 5 )
p2.generate.go( r, organism = NULL, go2all.egs = NULL, eg.alias2eg = NULL, min.env.length = 5 )
r |
a 'Pagoda2' object |
organism |
the organism (default=NULL). Currently 'hs' (human), 'mm' (mouse) and 'dr' (zebrafish) are supported. |
go2all.egs |
mappings between a given GO identifier and all of the Entrez Gene identifiers annotated at that GO term or to one of its child nodes in the GO ontology (default=NULL) |
eg.alias2eg |
mappings between common gene symbol identifiers and entrez gene identifiers (default=NULL) |
min.env.length |
numeric Minimum environment length (default=5) |
Generate a GO environment for human for overdispersion analysis for the the back end
p2.generate.human.go(r)
p2.generate.human.go(r)
r |
a 'Pagoda2' object |
a GO environment object
Generate a GO environment for mouse for overdispersion analysis for the the back end
p2.generate.mouse.go(r)
p2.generate.mouse.go(r)
r |
a 'Pagoda2' object |
a GO environment object
Create 'PAGODA1' web application from a 'Pagoda2' object 'PAGODA1' found here, with 'SCDE': <https://www.bioconductor.org/packages/release/bioc/html/scde.html>
p2.make.pagoda1.app( p2, col.cols = NULL, row.clustering = NULL, title = "pathway clustering", zlim = NULL, embedding = NULL, inner.clustering = TRUE, groups = NULL, clusterType = NULL, embeddingType = NULL, veloinfo = NULL, type = "PCA", min.group.size = 1, batch.colors = NULL, n.cores = 10 )
p2.make.pagoda1.app( p2, col.cols = NULL, row.clustering = NULL, title = "pathway clustering", zlim = NULL, embedding = NULL, inner.clustering = TRUE, groups = NULL, clusterType = NULL, embeddingType = NULL, veloinfo = NULL, type = "PCA", min.group.size = 1, batch.colors = NULL, n.cores = 10 )
p2 |
'Pagoda2' object |
col.cols |
Matrix of column colors (default=NULL). Useful for visualizing cell annotations such as batch labels. |
row.clustering |
Row dendrogram (default=NULL) |
title |
character Title to use (default="pathway clustering") |
zlim |
Range of the normalized gene expression levels (default=NULL). Input as a list: c(lower_bound, upper_bound). Values outside this range will be Winsorized. Useful for increasing the contrast of the heatmap visualizations. If NULL, set to the 5th and 95th percentiles. |
embedding |
A 2-D embedding of the cells (PCA, tSNE, etc.), passed as a data frame with two columns (two dimensions) and rows corresponding to cells (row names have to match cell names) (default=NULL). |
inner.clustering |
boolean Whether to get overall cell clustering (default=TRUE). |
groups |
factor describing grouping of different cells. If provided, the cross-fits and the expected expression magnitudes will be determined separately within each group. The factor should have the same length as ncol(counts) (default=NULL). |
clusterType |
cluster type (default=NULL). If NULL, takes the latest cluster in the 'Pagoda2' object using 'p2$clusters[[type]][[1]]' |
embeddingType |
embedding type (default=NULL). If NULL, takes the latest embedding in the 'Pagoda2' object using p2$embeddings[[type]][[1]] |
veloinfo |
cell velocity information, cell velocities (grid and cell) (default=NULL) |
type |
character Either 'counts' or a name of a 'reduction' in the 'Pagoda2' object (default='PCA') |
min.group.size |
integer Minimum group size (default=1) |
batch.colors |
colors of the batches, i.e. the factor (corresponding to rows of the model matrix) specifying batch assignment of each cell(default=NULL) |
n.cores |
numeric Number of cores (default=10) |
'PAGODA1' web application
Generate a list metadata structure that can be passed to a 'pagoda2' web object constructor as additional metadata given a named factor
p2.metadata.from.factor( metadata, displayname = NULL, s = 1, v = 1, start = 0, end = NULL, pal = NULL )
p2.metadata.from.factor( metadata, displayname = NULL, s = 1, v = 1, start = 0, end = NULL, pal = NULL )
metadata |
named factor with metadata for individual cells, names must correspond to cells |
displayname |
character Name to display for the metadata (default=NULL) |
s |
numeric Value for rainbow palette (default=1) |
v |
numeric Value for rainbow palette (default=1) |
start |
numeric Starting value (default=0) |
end |
numeric Ending value (default=NULL) |
pal |
optional vector of colours to use, if provided overrides s,v,start and end parameters (default=NULL) |
list of data, levels, palette to be passed to 'pagoda2' web object constructor
Generate a 'pagoda2' web object from a 'Pagoda2' object using hierarchical differential expression
p2.toweb.hdea(p2, title = "")
p2.toweb.hdea(p2, title = "")
p2 |
p2 object |
title |
character Name of the pagoda object (default="") |
a 'pagoda2' web object
Modified 'PAGODA1' app (from 'SCDE') for browsing 'pagoda2' results. Refer to 'ViewPagodaAppOld' and 'make.pagoda.app()' in 'SCDE'
results
Result object returned by scde.expression.difference()
(default=NULL). Note to browse group posterior levels, use return.posteriors = TRUE
in the scde.expression.difference()
call.
type
Either 'counts' or a name of a 'reduction' in the 'Pagoda2' object
genes
List of genes to display in the Detailed clustering panel (default=list())
batch
Any batch or other known confounders to be included in the visualization as a column color track (default=NULL)
pathways
character vector Pathway or gene names (default=NULL)
name
App name (needs to be altered only if adding more than one app to the server using the 'server' parameter) (default=NULL)
trim
Trim quantity used for Winsorization for visualization
embedding
Embedding information (default=NULL)
veloinfo
Velocity information (default=NULL)
goenv
environment mapping pathways to genes (default=NULL)
renv
Global environment (default=NULL)
new()
Initialize p2ViewPagodaApp class
p2ViewPagodaApp$new( results, pathways, genes, goenv, batch = NULL, name = "pathway overdispersion", trim = 1.1/nrow(p2$counts), embedding = NULL, type, veloinfo = NULL )
results
Result object returned by scde.expression.difference()
. Note to browse group posterior levels, use return.posteriors = TRUE
in the scde.expression.difference()
call.
pathways
character vector Pathway or gene names (default=NULL)
genes
list Genes to display in the Detailed clustering panel (default=list())
goenv
Environment mapping pathways to genes (default=NULL)
batch
Any batch or other known confounders to be included in the visualization as a column color track (default=NULL)
name
string App name (needs to be altered only if adding more than one app to the server using the 'server' parameter) (default="pathway overdispersion")
trim
numeric Trim quantity used for Winsorization for visualization (default=1.1/nrow(p2$counts) whereby the 'counts' from the 'Pagoda2' object is the gene count matrix, normalized on total counts (default=NULL)
embedding
Embedding information (default=NULL)
type
Either 'counts' or a name of a 'reduction' in the 'pagoda2' object
veloinfo
Velocity information (default=NULL)
new 'p2ViewPagodaApp' object
getgenecldata()
Helper function to get the heatmap data for a given set of genes
p2ViewPagodaApp$getgenecldata(genes = NULL, gcl = NULL, ltrim = 0)
genes
character vector Gene names (default=NULL)
gcl
pathway or gene-weighted PCA (default=NULL). If NULL, uses tp2c.view.pathways(self$genes, self$results$p2, goenv=goenv, vhc=self$results$hvc, plot=FALSE, trim=ltrim, n.genes=Inf).
ltrim
numeric Winsorization trim that should be applied (default=0)
heatmap data for a given set of genes
call()
Call Rook application. Using client-side ExtJS framework and Inchlib HTML5 canvas libraries to create the graphical user interface for PAGODA
p2ViewPagodaApp$call(env)
env
The environment argument is a true R environment object which the application is free to modify. Please see the Rook documentation for more details.
modified 'PAGODA1' app
clone()
The objects of this class are cloneable with this method.
p2ViewPagodaApp$clone(deep = FALSE)
deep
Whether to make a deep clone.
Collapse aspects driven by the same combinations of genes. (Aspects are some pattern across cells e.g. sequencing depth, or PC corresponding to an undesired process such as ribosomal pathway variation.) Examines PC loading vectors underlying the identified aspects and clusters of aspects based on a product of loading and score correlation (raised to corr.power). Clusters of aspects driven by the same genes are determined based on the parameter "distance.threshold".
pagoda.reduce.loading.redundancy( tam, pwpca, clpca = NULL, plot = FALSE, cluster.method = "complete", distance.threshold = 0.01, corr.power = 4, abs = TRUE, n.cores = 1, ... )
pagoda.reduce.loading.redundancy( tam, pwpca, clpca = NULL, plot = FALSE, cluster.method = "complete", distance.threshold = 0.01, corr.power = 4, abs = TRUE, n.cores = 1, ... )
tam |
output of pagoda.top.aspects(), i.e. a list structure containing the following items: xv: a matrix of normalized aspect patterns (rows: significant aspects, columns: cells) xvw: corresponding weight matrix gw: set of genes driving the significant aspects df: text table with the significance testing results |
pwpca |
output of pagoda.pathway.wPCA(), i.e. a list of weighted PCA info for each valid gene set |
clpca |
output of pagoda.gene.clusters() (optional) (default=NULL). The output of pagoda.gene.clusters() is a list structure containing the following fields: clusters: alist of genes in each cluster values xf: extreme value distribution fit for the standardized lambda1 of a randomly generated pattern tci: index of a top cluster in each random iteration cl.goc: weighted PCA info for each real gene cluster varm: standardized lambda1 values for each randomly generated matrix cluster clvlm: a linear model describing dependency of the cluster lambda1 on a Tracy-Widom lambda1 expectation |
plot |
boolean Whether to plot the resulting clustering (default=FALSE) |
cluster.method |
string One of the standard clustering methods to be used (default="complete") |
distance.threshold |
numeric Similarity threshold for grouping interdependent aspects (default=0.01) |
corr.power |
numeric Power to which the product of loading and score correlation is raised (default=4) |
abs |
boolean Whether to use absolute correlation (default=TRUE) |
n.cores |
numeric Number of cores to use during processing (default=1) |
... |
additional arguments are passed to the pagoda.view.aspects() method during plotting |
a list structure analogous to that returned by pagoda.top.aspects(), but with addition of a $cnam element containing a list of aspects summarized by each row of the new (reduced) $xv and $xvw
Collapse aspects driven by similar patterns (i.e. separate the same sets of cells) Examines PC loading vectors underlying the identified aspects and clusters aspects based on score correlation. Clusters of aspects driven by the same patterns are determined based on the distance.threshold.
pagoda.reduce.redundancy( tamr, distance.threshold = 0.2, cluster.method = "complete", distance = NULL, weighted.correlation = TRUE, plot = FALSE, top = Inf, trim = 0, abs = FALSE, ... )
pagoda.reduce.redundancy( tamr, distance.threshold = 0.2, cluster.method = "complete", distance = NULL, weighted.correlation = TRUE, plot = FALSE, top = Inf, trim = 0, abs = FALSE, ... )
tamr |
Combined pathways that show similar expression patterns, output of pagoda.reduce.loading.redundancy() |
distance.threshold |
numeric Similarity threshold for grouping interdependent aspects (default=0.2) |
cluster.method |
character One of the standard clustering methods to be used (default="complete") |
distance |
distance matrix (default=NULL) |
weighted.correlation |
boolean Whether to use a weighted correlation in determining the similarity of patterns (default=TRUE) |
plot |
boolean Whether to show plot (default=FALSE) |
top |
bololean Restrict output to the top N aspects of heterogeneity (default=Inf, i.e. no restriction) |
trim |
numeric Winsorization trim to use prior to determining the top aspects (default=0) |
abs |
boolean Whether to use absolute correlation (default=FALSE) |
... |
additional arguments are passed to the pagoda.view.aspects() method during plotting |
List structure analogous to that returned by pagoda.top.aspects(), but with addition of a $cnam element containing a list of aspects summarized by each row of the new (reduced) $xv and $xvw
Serialise an R array to a JSON object
arr |
An array (default=NULL) |
Serialised version of the array in JSON, which includes dimension information as separate fields
Parse pathways from originalP2object$misc$pathwayOD$xv into JSON
JSON with parsed cell order from mainDendrogram$cellorder
Handle httpd server calls
env |
The environment argument is a true R environment object which the application is free to modify. Please see the Rook documentation for more details. |
Parse cellmetadata into JSON
JSON with parsed cellmetadata
Parse mainDendrogram$cellorder into JSON
JSON with parsed cell order from mainDendrogram$cellorder
Parse originalP2object$misc$varinfo[,c("m","qv")] into JSON
JSON with parsed information from genename, dispersion, mean gene expression
Generate a dendrogram of groups
dendrogramCellGroups |
Cell groups to input into hclust() |
List of hcGroups, cellorder, and cluster.sizes
Generate information about the embeddings we are exporting
List with embeddings
Generate a JSON list representation of the gene kNN network
graph |
Input graph |
JSON with gene kNN network
Compress the embedding
reduc |
reduction |
embed |
embedding |
compressed embedding as JSON
Compress float64 array
v |
float64 array |
compressed array
Compress int32 array
v |
int32 array |
compressed array
Read a static file from the filesystem, and put in the response
filename |
path to filename |
Content to display or error page
Parse dendrogram into JSON
JSON with parsed dendrogram
Convert serialized file to static file
binary.filename |
path to binary file (default=NULL) |
verbose |
boolean Whether to give verbose output (default=FALSE) |
static file written by WriteListToBinary(expL=exportList, outfile=binary.filename, verbose=verbose)
Logging function for console
message |
Message to output for the console |
printed message
Create simple List from sparse Matrix with Dimnames as JSON
matsparse |
Sparse matrix |
List with slots i, p, x
pagoda2WebApp class to create 'pagoda2' web applications via a Rook server
originalP2object
Input 'Pagoda2' object
name
string Display name for the application
mat
Embedding
cellmetadata
Metadata associated with 'Pagoda2' object
mainDendrogram
Dendrogram from hclust() of all cells in the 'Pagoda2' object
geneSets
Gene sets in the 'Pagoda2' object
rookRoot
Rook server root directory
appmetadata
pagoda2 web application metadata
Calculate correlation distance between PC magnitudes given a number of target dimensions
pathway.pc.correlation.distance(pcc, xv, n.cores = 1, target.ndf = NULL)
pathway.pc.correlation.distance(pcc, xv, n.cores = 1, target.ndf = NULL)
pcc |
weighted PC magnitudes e.g. scde::pagoda.pathway.wPCA() gives the weighted PC magnitudes for each gene provided; e.g. scde::pagoda.gene.clusters() gives the weighted PC magnitudes for de novo gene sets identified by clustering on expression |
xv |
a matrix of normalized aspect patterns (rows: significant aspects, columns: cells) |
n.cores |
numeric Number of cores to use (default=1) |
target.ndf |
numeric Target dimensions (default=NULL) |
correlation distance matrix, akin to stats dist
Plot multiclassified cells per selection as a percent barplot
plotMulticlassified(sel)
plotMulticlassified(sel)
sel |
pagoda2 selection object |
ggplot2 object
Plot the embedding of a 'Pagoda2' object with the given values
plotOneWithValues( p2obj, values, title = "", type = "PCA", embeddingType = "tSNE" )
plotOneWithValues( p2obj, values, title = "", type = "PCA", embeddingType = "tSNE" )
p2obj |
the 'Pagoda2' object |
values |
the values to plot, fed into p2obj$plotEmbedding(colors=values) |
title |
character Title for the plot (default="") |
type |
character Type reduction on which the embedding is based on (default="PCA") |
embeddingType |
character Type of embedding to plot (default="tSNE") |
NULL, simply updates p2obj$plotEmbedding()
Get a dataframe and plot summarising overlaps between selection of a pagoda2 selection object ignore self overlaps
plotSelectionOverlaps(sel)
plotSelectionOverlaps(sel)
sel |
a pagoda2 selection object |
a list that contains a ggplot2 object and a datatable with the overlaps data
Takes as input a sparse matrix of the edge weights connecting each node to its nearest neighbors, and outputs a matrix of coordinates embedding the inputs in a lower-dimensional space.
projectKNNs( wij, dim = 2, sgd_batches = NULL, M = 5, gamma = 7, alpha = 1, rho = 1, coords = NULL, useDegree = FALSE, momentum = NULL, seed = NULL, threads = NULL, verbose = getOption("verbose", TRUE) )
projectKNNs( wij, dim = 2, sgd_batches = NULL, M = 5, gamma = 7, alpha = 1, rho = 1, coords = NULL, useDegree = FALSE, momentum = NULL, seed = NULL, threads = NULL, verbose = getOption("verbose", TRUE) )
wij |
A symmetric sparse matrix of edge weights, in C-compressed format, as created with the |
dim |
numeric The number of dimensions for the projection space (default=2) |
sgd_batches |
numeric The number of edges to process during SGD (default=NULL). Defaults to a value set based on the size of the dataset. If the parameter given is
between |
M |
numeric (largeVis) The number of negative edges to sample for each positive edge (default=5). |
gamma |
numeric (largeVis) The strength of the force pushing non-neighbor nodes apart (default=7). |
alpha |
numeric (largeVis) The hyperparameter in the distance function (default=1). The default distance function, |
rho |
(largeVis) numeric Initial learning rate (default=1) |
coords |
An initialized coordinate matrix (default=NULL) |
useDegree |
boolean Whether to use vertex degree to determine weights in negative sampling (if TRUE) or the sum of the vertex's edges (if FALSE) (default=FALSE) |
momentum |
If not NULL, SGD with momentum is used, with this multiplier, which must be between 0 and 1 (default=NULL). Note that momentum can drastically speed-up training time, at the cost of additional memory consumed. |
seed |
numeric Random seed to be passed to the C++ functions (default=NULL). Sampled from hardware entropy pool if |
threads |
numeric The maximum number of threads to spawn (default=NULL). Determined automatically if |
verbose |
boolean Verbosity (default=getOption("verbose", TRUE)) |
The algorithm attempts to estimate a dim
-dimensional embedding using stochastic gradient descent and
negative sampling.
The objective function is:
where is a probabilistic function relating the distance between two points in the low-dimensional projection space,
and the probability that they are nearest neighbors.
The default probabilistic function is . If
is set to zero,
an alternative probabilistic function,
will be used instead.
Note that the input matrix should be symmetric. If any columns in the matrix are empty, the function will fail.
A dense [N,D] matrix of the coordinates projecting the w_ij matrix into the lower-dimensional space.
If specified, seed
is passed to the C++ and used to initialize the random number generator. This will not, however, be
sufficient to ensure reproducible results, because the initial coordinate matrix is generated using the R
random number generator.
To ensure reproducibility, call set.seed
before calling this function, or pass it a pre-allocated coordinate matrix.
The original paper called for weights in negative sampling to be calculated according to the degree of each vertex, the number of edges connecting to the vertex. The reference implementation, however, uses the sum of the weights of the edges to each vertex. In experiments, the difference was imperceptible with small (MNIST-size) datasets, but the results seems aesthetically preferrable using degree. The default is to use the edge weights, consistent with the reference implementation.
Quick loading of 10X CellRanger count matrices
read.10x.matrices(matrixPaths, version = "V3", n.cores = 1, verbose = TRUE)
read.10x.matrices(matrixPaths, version = "V3", n.cores = 1, verbose = TRUE)
matrixPaths |
a single path to the folder containing matrix.mtx, genes.tsv and barcodes.tsv files, OR a named list of such paths |
version |
string Version of 10x output to read (default='V3'). Must be one of 'V2' or 'V3'. |
n.cores |
numeric Cores to utilize in parallel (default=1) |
verbose |
boolean Whether to output verbose output (default=TRUE) |
a sparse matrix representation of the data (or a list of sparse matrices if a list of paths was passed)
This function reads a matrix generated by the 10x processing pipeline from the specified directory and returns it. It aborts if one of the required files in the specified directory do not exist.
read10xMatrix(path, version = "V3", transcript.id = "SYMBOL", verbose = TRUE)
read10xMatrix(path, version = "V3", transcript.id = "SYMBOL", verbose = TRUE)
path |
string Location of 10x output |
version |
string Version of 10x output to read (default='V3'). Must be one of 'V2' or 'V3'. |
transcript.id |
string Transcript identifier to use (default='SYMBOL'). Must be either 'SYMBOL' (e.g. "Sox17") or 'ENSEMBL' (e.g. "ENSMUSG00000025902"). This value is case-sensitive. |
verbose |
boolean Whether to return verbose output |
parsed 10x outputs into a matrix
Read a pagoda2 cell selection file and return it as a factor while removing any mutliclassified cells
readPagoda2SelectionAsFactor(filepath, use.internal.name = FALSE)
readPagoda2SelectionAsFactor(filepath, use.internal.name = FALSE)
filepath |
name of the selection file |
use.internal.name |
boolean Use field 'internal.name' as factor names (default=FALSE). Passed to factorFromP2Selection |
a name factor with the membership of all the cells that are not multiclassified
Reads a 'pagoda2' web app exported cell selection file exported as a list of list objects that contain the name of the selection, the color (as a hex string) and the identifiers of the individual cells
readPagoda2SelectionFile(filepath)
readPagoda2SelectionFile(filepath)
filepath |
the path of the file load |
Remove cells that are present in more than one selection from all the selections they are in
removeSelectionOverlaps(selections)
removeSelectionOverlaps(selections)
selections |
a pagoda2 selections list |
a new list with the duplicated cells removed
Score cells by getting mean expression of genes in signatures
score.cells.nb0(data, signature)
score.cells.nb0(data, signature)
data |
matrix |
signature |
the genes in the signature |
cell scores
Puram, Bernstein (Cell, 2018) Score cells as described in Puram, Bernstein (Cell, 2018)
score.cells.puram(data, signature, correct = TRUE, show.plot = FALSE, ...)
score.cells.puram(data, signature, correct = TRUE, show.plot = FALSE, ...)
data |
matrix of expression, rows are cell, columns are genes |
signature |
character vector The signature to evaluate, a character vector of genes |
correct |
boolean Perform background correction by getting a semi-random geneset (default=TRUE) |
show.plot |
boolean If corrected values are calculated show plot of corrected vs original scores (default=FALSE) |
... |
options for get.control.geneset() |
a score for each cell
in the original paper.Calculate the default number of batches for a given number of vertices and edges.
The formula used is the one used by the 'largeVis' reference implementation. This is substantially less than the recommendation in the original paper.
sgdBatches(N, E = 150 * N/2)
sgdBatches(N, E = 150 * N/2)
N |
Number of vertices |
E |
Number of edges (default = 150*N/2) |
The recommended number of sgd batches.
# Observe that increasing K has no effect on processing time N <- 70000 # MNIST K <- 10:250 plot(K, sgdBatches(rep(N, length(K)), N * K / 2)) # Observe that processing time scales linarly with N N <- c(seq(from = 1, to = 10000, by = 100), seq(from = 10000, to = 10000000, by = 1000)) plot(N, sgdBatches(N))
# Observe that increasing K has no effect on processing time N <- 70000 # MNIST K <- 10:250 plot(K, sgdBatches(rep(N, length(K)), N * K / 2)) # Observe that processing time scales linarly with N N <- c(seq(from = 1, to = 10000, by = 100), seq(from = 10000, to = 10000000, by = 1000)) plot(N, sgdBatches(N))
Directly open the 'pagoda2' web application and view the 'p2web' application object from our R session
show.app(app, name, port, ip, browse = TRUE, server = NULL)
show.app(app, name, port, ip, browse = TRUE, server = NULL)
app |
'pagoda2' application object |
name |
character Name of the application to view |
port |
numeric Port number |
ip |
numeric IP address |
browse |
boolean Whether to load the app into an HTML browser (default=TRUE) |
server |
server If NULL, will grab server with get.scde.server(port=port, ip=ip) (derfault=NULL) |
application within browser
Subset a gene signature to the genes in the given matrix with optional warning if genes are missing
subsetSignatureToData(data, signature, raise.warning = TRUE)
subsetSignatureToData(data, signature, raise.warning = TRUE)
data |
matrix |
signature |
character vector The gene signature from which to subset a character vector of genes |
raise.warning |
boolean Warn if genes are missing (default=TRUE) |
The filtered subset of gene signatures
View pathway or gene-weighted PCA 'Pagoda2' version of the function pagoda.show.pathways() Takes in a list of pathways (or a list of genes), runs weighted PCA, optionally showing the result.
tp2c.view.pathways( pathways, p2, goenv = NULL, batch = NULL, n.genes = 20, two.sided = TRUE, n.pc = rep(1, length(pathways)), colcols = NULL, zlim = NULL, labRow = NA, vhc = NULL, cexCol = 1, cexRow = 1, nstarts = 50, row.order = NULL, show.Colv = TRUE, plot = TRUE, trim = 1.1/nrow(p2$counts), showPC = TRUE, ... )
tp2c.view.pathways( pathways, p2, goenv = NULL, batch = NULL, n.genes = 20, two.sided = TRUE, n.pc = rep(1, length(pathways)), colcols = NULL, zlim = NULL, labRow = NA, vhc = NULL, cexCol = 1, cexRow = 1, nstarts = 50, row.order = NULL, show.Colv = TRUE, plot = TRUE, trim = 1.1/nrow(p2$counts), showPC = TRUE, ... )
pathways |
character vector of pathway or gene names |
p2 |
'Pagoda2' object |
goenv |
environment mapping pathways to genes (default=NULL) |
batch |
factor (corresponding to rows of the model matrix) specifying batch assignment of each cell, to perform batch correction (default=NULL). |
n.genes |
integer Number of genes to show (default=20) |
two.sided |
boolean If TRUE, the set of shown genes should be split among highest and lowest loading (default=TRUE). If FALSE, genes with highest absolute loading should be shown. |
n.pc |
integer vector Number of principal component to show for each listed pathway(default=rep(1, length(pathways))) |
colcols |
column color matrix (default=NULL) |
zlim |
numeric z color limit (default=NULL) |
labRow |
row labels (default=NA) |
vhc |
cell clustering (default=NULL) |
cexCol |
positive numbers, used as cex.axis in for the row or column axis labeling(default=1) |
cexRow |
positive numbers, used as cex.axis in for the row or column axis labeling(default=1) |
nstarts |
integer Number of random starts to use (default=50) |
row.order |
row order (default=NULL). If NULL, uses order from hclust. |
show.Colv |
boolean Whether to show cell dendrogram (default=TRUE) |
plot |
boolean Whether to plot (default=TRUE) |
trim |
numeric Winsorization trim that should be applied (default=1.1/nrow(p2$counts)). Note that p2 is a 'Pagoda2' object. |
showPC |
boolean (default=TRUE) |
... |
parameters to pass to my.heatmap2. Only if plot is TRUE. |
cell scores along the first principal component of shown genes (returned as invisible)
Validates a pagoda2 selection object
validateSelectionsObject(selections)
validateSelectionsObject(selections)
selections |
the pagoda2 selection object to be validated |
a logical value indicating if the object is valid
Generate a 'pagoda2' web object
webP2proc( p2, additionalMetadata = NULL, title = "Pagoda2", make.go.sets = TRUE, make.de.sets = TRUE, go.env = NULL, make.gene.graph = TRUE, appmetadata = NULL )
webP2proc( p2, additionalMetadata = NULL, title = "Pagoda2", make.go.sets = TRUE, make.de.sets = TRUE, go.env = NULL, make.gene.graph = TRUE, appmetadata = NULL )
p2 |
a 'Pagoda2' object |
additionalMetadata |
'pagoda2' web metadata object (default=NULL) |
title |
character string Title for the web app (default='Pagoda2') |
make.go.sets |
boolean Whether GO sets should be made (default=TRUE) |
make.de.sets |
boolean Whether differential expression sets should be made (default=TRUE) |
go.env |
the GO environment used for the overdispersion analysis (default=NULL) |
make.gene.graph |
logical specifying if the gene graph should be make, if FALSE the find similar genes functionality will be disabled on the web app |
appmetadata |
'pagoda2' web application metadata (default=NULL) |
a 'pagoda2' web application
Sets the ncol(mat)*trim top outliers in each row to the next lowest value same for the lowest outliers
winsorize.matrix(mat, trim)
winsorize.matrix(mat, trim)
mat |
Numeric matrix |
trim |
numeric Fraction of outliers (on each side) that should be Winsorized, or (if the value is >= 1) the number of outliers to be trimmed on each side |
Winsorized matrix
set.seed(0) mat <- matrix( c(rnorm(5*10,mean=0,sd=1), rnorm(5*10,mean=5,sd=1)), 10, 10) # random matrix mat[1,1] <- 1000 # make outlier range(mat) # look at range of values win.mat <- winsorize.matrix(mat, 0.1) range(win.mat) # note outliers removed
set.seed(0) mat <- matrix( c(rnorm(5*10,mean=0,sd=1), rnorm(5*10,mean=5,sd=1)), 10, 10) # random matrix mat[1,1] <- 1000 # make outlier range(mat) # look at range of values win.mat <- winsorize.matrix(mat, 0.1) range(win.mat) # note outliers removed
Writes a list of genes as a gene selection that can be loaded in the web interface
writeGenesAsPagoda2Selection(name, genes, filename)
writeGenesAsPagoda2Selection(name, genes, filename)
name |
the name of the selection |
genes |
a string vector of the gene names |
filename |
the filename to save to |
NULL, writes to filepath the list of genes as a gene selection that can be loaded in the web interface
Writes a pagoda2 selection object as a p2 selection file that be be loaded to the web interface
writePagoda2SelectionFile(sel, filepath)
writePagoda2SelectionFile(sel, filepath)
sel |
pagoda2 selection object |
filepath |
name of file to which to write |
NULL, writes to filepath the pagoda2 selection object as a p2 selection file that be be loaded to the web interface