---
title: "Introduction to sparsenetgls"
author: "Irene SL Zeng and Thomas Lumley"
package: sparsenetgls
output: 
    BiocStyle::html_document:
    toc: true

vignette: >
    %\VignetteIndexEntry{Introduction to sparsenetgls}
    %\VignetteEngine{knitr::rmarkdown}
    %\VignetteEncoding{UTF-8}  
---

```{r setup, include=TRUE}
library(knitr)
library(rmarkdown)
knitr::opts_chunk$set(echo = TRUE)
```

# Introduction

## Summary  
When we use regression models for Gaussian multivariate data curated from 
high-throughput assays("omic" data), the response variables normally have 
their latent structure in the variance-covariance matrix and its inverse. 
The conventional approach gives assumptions of these structures in regression.
For example, in repeated measure analysis of variance, the covariance matrix 
is given a fixed form of the structure.  However, these assumptions are not 
known to be true unless there are prior information available. 
Accurate information of the variance covariance matrix and the precision matrix
will achieve better estimation in the regression coefficients.  In cases of 
response variables with only a small numbers of non-zero covariance
terms in the variance-covariance matrix, they are equivalent to a sparse 
undirected network graph of which nodes represent variables and edges represent
non-zero links between these variables.

Our method is to utilize the Gaussian Graphical Model (GGM) in estimating the 
structure of a sparse variance-covariance matrix and its inverse (precision 
matrix), in order to improve the estimation of the multivariate generalized 
least square regression.   

The method firstly uses lasso penalization method "glasso", neighbour 
selection method or "enet" method to estimate the precision matrix of 
these response variables; and Secondly selects the covariance terms from the 
sample variance-covariance matrix based on an estimated graph structure of the 
precision matrix and a fine-tuning parameter. The fine tuning parameter is 
selected according to the primitive graph theory which bases on power of the 
adjacent matrix to infer the final structure of the variance-covariance matrix.


## The model 
The sparsenetgls is a multivariate regression model given an estimated 
precision matrix and variance-covariance matrix from the response variables. 
It uses the sandwich estimator of the variance-covariance matrix of the 
regression coefficients. 
In the estimation for the precision matrix, there are four options provided in 
the package sparsenetgls and they are "glasso", "lasso", "mb", and "elastic". 
"glasso" used the graphical lasso method proposed by Yuan and Lin (2007), 
and the block-wise coordinate descend algorithm by Friedman, Hastie, Hofling 
and Tibshirani (2007).  
"lasso" and "mb" use the linear regression with lasso penalization to provide 
an asymmetric approximation which is proposed by  Meinshausen and 
Buhlmann (2006). 
"enet" is also an asymmetric approximation using linear regressions but using 
the combination of lasso penalization (l1) and ridge regularization(l2).   
In the generalized least square regression, regression coefficients are 
estimated from the multilevel formatted multivariate model. Response variables 
are stacked into one univariate regression variable with an indicator to 
identify their sampling units. The sandwich estimator of the variance is 
calculated based on the given precision, and variance-covariance matrix.    
There are different options for the selection of the regression coefficients 
from the penalized path. One is to base on the minimal variance, the other 
options are the information critiria (AIC and BIC).  The variances of the 
selected regression coefficients are calculated from its sandwich estimator 
given the selected structure of the precision matrix and the selected 
covariance terms based on the fine-tuning parameter.  


# The sparsenetgls R package 

The main functions included in the package "sparsenetgls" are:
    
1. path_result_for_roc(): the path_result_for_roc function is designed to 
produce the diagnosis index for the prediction accuracy of a Gaussian Graphical
model (GGM) when comparing it to the true graph structure.  The GGM must use a
L-p norm regularizations (p=1,2) with the series of solutions conditional on 
theregularization parameter.  The returning list of assessment results for a 
series of precision matrices includes sensitiviy/specificity/Nagative 
predicitedvalue /Positive predicted value. 
    
2. plot_roc(): it is designed to produce the Receiver Operative 
Characteristics (ROC) Curve to visualize the prediction accuracy of a Gaussian 
Graphical model (GGM) to the true graph structure. The GGM must use a L-p norm 
regularizations (p=1,2) with the series of solutions conditional on the 
regularization parameter.
    
    
3. sparsenetgls(): it is designed for multivariate regression given a 
penalized and/or regularised precision and covariance Matrix of the 
multivariate Gaussian distributed responses. Generalized least squared 
regression is used to derive the sandwich variances-covariance matrix for the 
regression coefficients of the explanatory variables, conditional on the 
solutions of the precision and covariance matrix.
    
    
4. plotsngls(): It is designed to provide the line plots of penalized 
parameter lambda and variance of regression coefficients in gls regression. It 
also provides the graph structures of the solutions to the precision matrix in 
the penalized path.


# Installation

There are two options for the installation: 

## Use bioconductor as installation source
```{r installation1,eval=FALSE}
install.packages("BiocManager")
BiocManaged::install("sparsenetgls")
library(sparsenetgls)
```

## Use github as installation source
```{r installation2,eval=FALSE}
devtools::install_github("superOmics/sparsenetgls")
library(sparsenetgls)
```

# Instructions and Examples of using the main functions

The following section provides examples using a known precision matrix 
stored in R datafile: bandprec.rdata.
The examples include:
    
1) To assess the results from Gaussian graphical model given by sparsenetgls;

2) Use sparsenetgls to estimate the regression coefficients from a multivariate
regression and use the minimal variance , information criteria to select the 
regression coefficients;
    
3) Visualize the results from sparsenetgls;

4) Use different options: "lasso", "mb", and "elastic" for the GGM in gls. 

Example 1 includes codes for assessing the results of a GGM from the 
sparsenetgls function. 
The first 12 lines is to simulate a dataset of response variables given the 
known precision matrix and a set of explanatory variables. The returning list 
of the sparsenetgls function includes the series of precision matrix produced 
from one of the GGM option (specified by method).
In plot_roc , both of the ngroup and group option indicate if the assessed 
results are for a group of GGM or for only one series of GGM. 

```{r example 1: assess estimated result from a GGM, eval=TRUE}
library(MASS)
library(Matrix)
library(sparsenetgls)

#simulate the dataset
data(bandprec, package="sparsenetgls")
varKnown <- solve(as.matrix(bandprec))
prec <- as.matrix(bandprec)
    
Y0 <- mvrnorm(n=100, mu=rep(0,50), Sigma=varKnown)
nlambda=10                                     
#u-beta
u <- rep(1,8)
X_1 <- mvrnorm(n=100, mu=rep(0,8), Sigma=Diagonal(8,rep(1,8)))
Y_1 <- Y0+as.vector(X_1%*%as.matrix(u))
databand_X=X_1
databand_Y=Y_1

#produce the precision matrices
omega <- sparsenetgls(responsedata=databand_Y, predictdata=databand_X, 
nlambda=10, ndist=1, method="glasso")$PREC_seq
omega_est <- array(dim=c(50,50,10))
for (i in seq_len(10)) omega_est[,,i] <- as.matrix(omega[[i]])

roc_path_result <- path_result_for_roc(PREC_for_graph=prec, OMEGA_path=omega_est
, pathnumber=10)
plot_roc(result_assessment=roc_path_result, group=FALSE, ngroup=0,
est_names="glasso estimation")
```

Example 2 provides codes for deriving the regression coefficients from 
sparsenetgls using "glasso" method to estimate the precision matrix of GGM. 
The function convertbeta() is to convert the regression coefficients from the 
standardized scale to the original scale. 
The following codes after the convertion is to select betas with the minimal 
variance, aic or bic. 

```{r example 2: Use the sparsenetgls and convertbeta function,eval=TRUE}

fitgls <- sparsenetgls(databand_Y, databand_X, nlambda=10, 
ndist=5, method="glasso")

#Convert the regression coefficients to its original scale
q <- dim(databand_X)[2]
nlambda=10

betagls <- matrix(nrow=nlambda, ncol=q+1)
for (i in seq_len(nlambda))   
betagls[i,] <- convertbeta(Y=databand_Y, X=databand_X, q=q+1,
beta0=fitgls$beta[,i])$betaconv

#Beta selection 

#select lamda and dist value based on the minimal variance of beta
ndist <- max(fitgls$power)-1
tr_gamma <- matrix(nrow=10, ncol=ndist-1)

for (j in seq_len(ndist-1))
    for (i in seq_len(nlambda)) 
        tr_gamma[i,j] <- (sum(diag(fitgls$covBeta[,,j,i]))) 

select.lambda.dist <- which(tr_gamma==min(tr_gamma), arr.ind=TRUE)
select.lambda.dist
betagls_select <- betagls[select.lambda.dist[1],]

#row is lambda and column is dist
varbeta <- diag(fitgls$covBeta[,,ndist,select.lambda.dist[1]])

##select lamda and dist value based on the AIC and BIC
select.lambda.dist2 <- which(fitgls$bic==min(fitgls$bic,na.rm=TRUE),
arr.ind=TRUE)
select.lambda.dist3 <- which(fitgls$aic==min(fitgls$aic,na.rm=TRUE),
arr.ind=TRUE)
varbeta_bic <- diag(fitgls$covBeta[,,ndist,select.lambda.dist2[1]])
varbeta_aic <- diag(fitgls$covBeta[,,ndist,select.lambda.dist3[1]])

```

Example 3 is to visualize the results by line-plots and structure-plots for 
the derived precision matrix from the sparsenet object. 

```{r example 3, Get the visualized result, eval=TRUE}
plotsngls(fitgls,ith_lambda=5)
```

Example 4 provides examples for using different options in estimating GGM in 
sparsenetgls.

```{r example 4, Use different options of GGM estimation, eval=TRUE}
#Use the glasso method to estimate the precision matrix
fitgls_g <- sparsenetgls(databand_Y, databand_X, nlambda=10, ndist=5,
method="elastic")

#Uset the lasso method to approximate the precision matrix
#fitgls_l <- sparsenetgls(databand_Y, databand_X, nlambda=10, ndist=5,
#method="lasso")

#use the Meinshausen B?hlmann method to approximate the precision matrix
#fitgls_m <- sparsenetgls(databand_Y, databand_X, nlambda=10, ndist=5,
#method="mb")
```

# sessionInfo
All of the outputs in this vignette are produced under the following 
conditions:

```{r sessionInfo,echo=TRUE,eval=TRUE}
sessionInfo()
```

# References
1. Dempster, A.P. Covariance Selection. Biomatrics 1972;28(1):157-175.
2. Edwards, D. Introduction to Graphical Modelling. New York: Springer; 2000.
3. Friedman, J., Hastie, T., Simon, N. and Tibshiran, R. Regularization Paths 
for Generalized Linear Models via Coordinate Descent. Journal OF Statistical 
Software 2010;33(1).
4. Friedman, J., Hastie, T., Simon, N. and Tibshirani, R. 2016. Lasso and 
Elastic-Net Regularized Generalized Linear Models
5. Friedman, J., Hastie, T. and Tibshirani, R. Applications of the lasso and 
grouped lasso to the estimation of sparse graphical models. In. US; 2010.
6. Meinshausen, N. and Buhlmann, P. High-dimensional graphs and variable 
selection with the lasso. The annals of statistics 2006;34(3):1436-1462.
7. Pourahmadi, M. Covariance Estimation : The GLM and Regularization 
perspectives. Statistical Science 2011;26(3):369-387.
8. Yuan, M. and Lin, Y. Model Selection and Estimation in the Gaussian 
Graphical Model. Biometrika 2007;94(1.):19-35.
9. Zhao, T. and Liu, H. The huge Package for High-dimensional Undirected Graph
estimation in R. Journal of Machine Learning Research 2012;13: 1059-1062.
10. Zou, H. and Hastie, T. Regularization and variable selection via the 
Elastic net. Journal of the Royal Statistical Society 2005;2:301-320.
