Conversión de lista anidada a marco de datos

dataframe nested-lists

Question

El objetivo es convertir una lista anidada que a veces contiene registros faltantes en un marco de datos. Un ejemplo de la estructura cuando faltan registros es:

str(mylist)

List of 3
 $ :List of 7
  ..$ Hit    : chr "True"
  ..$ Project: chr "Blue"
  ..$ Year   : chr "2011"
  ..$ Rating : chr "4"
  ..$ Launch : chr "26 Jan 2012"
  ..$ ID     : chr "19"
  ..$ Dept   : chr "1, 2, 4"
 $ :List of 2
  ..$ Hit  : chr "False"
  ..$ Error: chr "Record not found"
 $ :List of 7
  ..$ Hit    : chr "True"
  ..$ Project: chr "Green"
  ..$ Year   : chr "2004"
  ..$ Rating : chr "8"
  ..$ Launch : chr "29 Feb 2004"
  ..$ ID     : chr "183"
  ..$ Dept   : chr "6, 8"

Cuando no faltan registros, la lista se puede convertir en un marco de datos usando data.frame(do.call(rbind.data.frame, mylist)) . Sin embargo, cuando faltan registros, esto da como resultado una columna no coincidente. Sé que hay funciones para combinar marcos de datos de columnas que no coinciden, pero aún no he encontrado una que se pueda aplicar a las listas. El resultado ideal sería mantener el registro 2 con NA para todas las variables. Esperando algo de ayuda.

Editar para agregar dput(mylist) :

list(structure(list(Hit = "True", Project = "Blue", Year = "2011", 
Rating = "4", Launch = "26 Jan 2012", ID = "19", Dept = "1, 2, 4"), .Names = c("Hit", 
"Project", "Year", "Rating", "Launch", "ID", "Dept")), structure(list(
Hit = "False", Error = "Record not found"), .Names = c("Hit", 
"Error")), structure(list(Hit = "True", Project = "Green", Year = "2004", 
Rating = "8", Launch = "29 Feb 2004", ID = "183", Dept = "6, 8"), .Names = c("Hit", 
"Project", "Year", "Rating", "Launch", "ID", "Dept")))

Answer 1

Puede crear una lista de data.frames:

dfs <- lapply(mylist, data.frame, stringsAsFactors = FALSE)

Luego usa uno de estos:

library(plyr)
rbind.fill(dfs)

o el más rápido

library(dplyr)
rbind_all(dfs)

En el caso de dplyr::rbind_all , me sorprende que elija usar "" lugar de NA para los datos faltantes. Si elimina stringsAsFactors = FALSE , obtendrá NA pero a costa de una advertencia ... Así que suppressWarnings(rbind_all(lapply(mylist, data.frame))) advertencias suppressWarnings(rbind_all(lapply(mylist, data.frame))) sería una solución fea pero rápida.

Answer 2

También puede usar (al menos v1.9.3) de rbindlist en el paquete data.table :

library(data.table)

rbindlist(mylist, fill=TRUE)

##      Hit Project Year Rating      Launch  ID    Dept            Error
## 1:  True    Blue 2011      4 26 Jan 2012  19 1, 2, 4               NA
## 2: False      NA   NA     NA          NA  NA      NA Record not found
## 3:  True   Green 2004      8 29 Feb 2004 183    6, 8               NA

Answer 3

Acabo de desarrollar una solución para esta pregunta que es aplicable aquí, así que la proporcionaré aquí también:

tl <- function(e) { if (is.null(e)) return(NULL); ret <- typeof(e); if (ret == ''list'' && !is.null(names(e))) ret <- list(type=''namedlist'') else ret <- list(type=ret,len=length(e)); ret; };
mkcsv <- function(v) paste0(collapse='','',v);
keyListToStr <- function(keyList) paste0(collapse='''',''/'',sapply(keyList,function(key) if (is.null(key)) ''*'' else paste0(collapse='','',key)));

extractLevelColumns <- function(
    nodes, ## current level node selection
    ..., ## additional arguments to data.frame()
    keyList=list(), ## current key path under main list
    sep=NULL, ## optional string separator on which to join multi-element vectors; if NULL, will leave as separate columns
    mkname=function(keyList,maxLen) paste0(collapse=''.'',if (is.null(sep) && maxLen == 1L) keyList[-length(keyList)] else keyList) ## name builder from current keyList and character vector max length across node level; default to dot-separated keys, and remove last index component for scalars
) {
    cat(sprintf(''extractLevelColumns(): %s/n'',keyListToStr(keyList)));
    if (length(nodes) == 0L) return(list()); ## handle corner case of empty main list
    tlList <- lapply(nodes,tl);
    typeList <- do.call(c,lapply(tlList,`[[`,''type''));
    if (length(unique(typeList)) != 1L) stop(sprintf(''error: inconsistent types (%s) at %s.'',mkcsv(typeList),keyListToStr(keyList)));
    type <- typeList[1L];
    if (type == ''namedlist'') { ## hash; recurse
        allKeys <- unique(do.call(c,lapply(nodes,names)));
        ret <- do.call(c,lapply(allKeys,function(key) extractLevelColumns(lapply(nodes,`[[`,key),...,keyList=c(keyList,key),sep=sep,mkname=mkname)));
    } else if (type == ''list'') { ## array; recurse
        lenList <- do.call(c,lapply(tlList,`[[`,''len''));
        maxLen <- max(lenList,na.rm=T);
        allIndexes <- seq_len(maxLen);
        ret <- do.call(c,lapply(allIndexes,function(index) extractLevelColumns(lapply(nodes,function(node) if (length(node) < index) NULL else node[[index]]),...,keyList=c(keyList,index),sep=sep,mkname=mkname))); ## must be careful to translate out-of-bounds to NULL; happens automatically with string keys, but not with integer indexes
    } else if (type%in%c(''raw'',''logical'',''integer'',''double'',''complex'',''character'')) { ## atomic leaf node; build column
        lenList <- do.call(c,lapply(tlList,`[[`,''len''));
        maxLen <- max(lenList,na.rm=T);
        if (is.null(sep)) {
            ret <- lapply(seq_len(maxLen),function(i) setNames(data.frame(sapply(nodes,function(node) if (length(node) < i) NA else node[[i]]),...),mkname(c(keyList,i),maxLen)));
        } else {
            ## keep original type if maxLen is 1, IOW don''t stringify
            ret <- list(setNames(data.frame(sapply(nodes,function(node) if (length(node) == 0L) NA else if (maxLen == 1L) node else paste(collapse=sep,node)),...),mkname(keyList,maxLen)));
        }; ## end if
    } else stop(sprintf(''error: unsupported type %s at %s.'',type,keyListToStr(keyList)));
    if (is.null(ret)) ret <- list(); ## handle corner case of exclusively empty sublists
    ret;
}; ## end extractLevelColumns()
## simple interface function
flattenList <- function(mainList,...) do.call(cbind,extractLevelColumns(mainList,...));

Ejecución:

## define data
mylist <- list(structure(list(Hit=''True'',Project=''Blue'',Year=''2011'',Rating=''4'',Launch=''26 Jan 2012'',ID=''19'',Dept=''1, 2, 4''),.Names=c(''Hit'',''Project'',''Year'',''Rating'',''Launch'',''ID'',''Dept'')),structure(list(Hit=''False'',Error=''Record not found''),.Names=c(''Hit'',''Error'')),structure(list(Hit=''True'',Project=''Green'',Year=''2004'',Rating=''8'',Launch=''29 Feb 2004'',ID=''183'',Dept=''6, 8''),.Names=c(''Hit'',''Project'',''Year'',''Rating'',''Launch'',''ID'',''Dept'')));

## run it
df <- flattenList(mylist);
## extractLevelColumns():
## extractLevelColumns(): Hit
## extractLevelColumns(): Project
## extractLevelColumns(): Year
## extractLevelColumns(): Rating
## extractLevelColumns(): Launch
## extractLevelColumns(): ID
## extractLevelColumns(): Dept
## extractLevelColumns(): Error

df;
##     Hit Project Year Rating      Launch   ID    Dept            Error
## 1  True    Blue 2011      4 26 Jan 2012   19 1, 2, 4             <NA>
## 2 False    <NA> <NA>   <NA>        <NA> <NA>    <NA> Record not found
## 3  True   Green 2004      8 29 Feb 2004  183    6, 8             <NA>

Mi función es más poderosa que data.table::rbindlist() desde 1.9.6, ya que puede manejar cualquier número de niveles de anidación y diferentes longitudes de vector en las ramas. En la pregunta vinculada, mi función aplana correctamente la lista del OP a un data.frame, pero data.table::rbindlist() falla con "Error in rbindlist(jsonRList, fill = T) : Column 4 of item 16 is length 2, inconsistent with first column of that item which is length 1. rbind/rbindlist doesn''t recycle as it already expects each item to be a uniform list, data.frame or data.table" .