Cómo importar datos CSV/TSV a Couch DB?

couchdb

Question

Cómo importar datos CSV / TSV a Couch DB?

Answer 1

Apache CouchDB solo almacena documentos JSON. Por lo tanto, para importar CSV debe convertir a documentos JSON individuales y luego POSTALIZARlos normalmente.

Probablemente tendrá que escribir un programa para recorrer cada línea. Convierta la línea CSV (una secuencia de valores ) en el documento JSON (una secuencia de pares clave: valor ). Entonces simplemente envíelo a CouchDB usando HTTP.

Answer 2

Es bastante fácil con Python.

 #!/usr/bin/env python

from couchdbkit import Server, Database
from couchdbkit.loaders import FileSystemDocsLoader
from csv import DictReader
import sys, subprocess, math, os



def parseDoc(doc):
    for k,v in doc.items():
        if (isinstance(v,str)):
            #print k, v, v.isdigit()
            # #see if this string is really an int or a float
            if v.isdigit()==True: #int
                doc[k] = int(v)
            else: #try a float
                try:
                    if math.isnan(float(v))==False:
                        doc[k] = float(v) 
                except:
                    pass            
    return doc


def upload(db, docs):
    db.bulk_save(docs)
    del docs
    return list()


def uploadFile(fname, uri, dbname):


  print ''Upload contents of %s to %s/%s'' % (fname, uri, dbname)

  # #connect to the db
  theServer = Server(uri)
  db = theServer.get_or_create_db(dbname)

  #loop on file for upload
  reader = DictReader(open(fname, ''rU''), dialect = ''excel'')  #see the python csv module 
         #for other options, such as using the tab delimeter. The first line in your csv 
         #file should contain all of the "key" and all subsequent lines hold the values 
         #for those keys.

  #used for bulk uploading
  docs = list()
  checkpoint = 100

  for doc in reader:
    newdoc = parseDoc(doc) #this just converts strings that are really numbers into ints and floats

    #Here I check to see if the doc is already on the database. If it is, then I assign
    #the _rev key so that it updates the doc on the db.

    if db.doc_exist(newdoc.get(''_id'')):
      newdoc[''_rev''] = db.get_rev(newdoc.get(''_id''))

    docs.append(newdoc)

    if len(docs)%checkpoint==0:
      docs = upload(db,docs)

  #don''t forget the last batch        
  docs = upload(db,docs)



if __name__==''__main__'':
  filename = sys.argv[1]
  uri = sys.argv[2]
  dbname = sys.argv[3]

  uploadFile(filename, uri, dbname)

Answer 3

Acabo de escribir un guión en Ruby: csv2couchdb

Answer 4

Yo uso esto aquí: https: // github / glynnbird / couchimport . Si su CSV es razonablemente simple, simplemente se reducirá a establecer un nombre de base de datos y conectar el CSV a couchimport.