machine-learning - example - tensorflow tutorial

Keras, problema escaso de la matriz

Question

Estoy tratando de alimentar una gran matriz dispersa con el modelo de Keras. Como el conjunto de datos no encaja en la memoria RAM, el camino a seguir es capacitar al modelo en un lote generado por lote de datos por un generador.

Para probar este enfoque y asegurarme de que mi solución funciona bien, modifiqué ligeramente un simple MLP de Kera en la tarea de clasificación de temas de Reuters Newswire . Entonces, la idea es comparar modelos originales y editados. Simplemente convierto numpy.ndarray en scipy.sparse.csr.csr_matrix y lo paso al modelo.

Pero mi modelo falla en algún momento y necesito una mano para descubrir una razón.

Aquí está el modelo original y mis adiciones a continuación

from __future__ import print_function
import numpy as np
np.random.seed(1337) # for reproducibility
from keras.datasets import reuters
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation
from keras.utils import np_utils
from keras.preprocessing.text import Tokenizer
max_words = 1000
batch_size = 32
nb_epoch = 5
print(''Loading data...'')
(X_train, y_train), (X_test, y_test) = reuters.load_data(nb_words=max_words, test_split=0.2)
print(len(X_train), ''train sequences'')
print(len(X_test), ''test sequences'')
nb_classes = np.max(y_train)+1
print(nb_classes, ''classes'')
print(''Vectorizing sequence data...'')
tokenizer = Tokenizer(nb_words=max_words)
X_train = tokenizer.sequences_to_matrix(X_train, mode=''binary'')
X_test = tokenizer.sequences_to_matrix(X_test, mode=''binary'')
print(''X_train shape:'', X_train.shape)
print(''X_test shape:'', X_test.shape)
print(''Convert class vector to binary class matrix (for use with categorical_crossentropy)'')
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)
print(''Y_train shape:'', Y_train.shape)
print(''Y_test shape:'', Y_test.shape)
print(''Building model...'')
model = Sequential()
model.add(Dense(512, input_shape=(max_words,)))
model.add(Activation(''relu''))
model.add(Dropout(0.5))
model.add(Dense(nb_classes))
model.add(Activation(''softmax''))
model.compile(loss=''categorical_crossentropy'',
optimizer=''adam'',
metrics=[''accuracy''])
history = model.fit(X_train, Y_train,
nb_epoch=nb_epoch, batch_size=batch_size,
verbose=1)#, validation_split=0.1)
#score = model.evaluate(X_test, Y_test,
# batch_size=batch_size, verbose=1)
print(''Test score:'', score[0])
print(''Test accuracy:'', score[1])

Emite:

Loading data...
8982 train sequences
2246 test sequences
46 classes
Vectorizing sequence data...
X_train shape: (8982, 1000)
X_test shape: (2246, 1000)
Convert class vector to binary class matrix (for use with categorical_crossentropy)
Y_train shape: (8982, 46)
Y_test shape: (2246, 46)
Building model...
Epoch 1/5
8982/8982 [==============================] - 5s - loss: 1.3932 - acc: 0.6906
Epoch 2/5
8982/8982 [==============================] - 4s - loss: 0.7522 - acc: 0.8234
Epoch 3/5
8982/8982 [==============================] - 5s - loss: 0.5407 - acc: 0.8681
Epoch 4/5
8982/8982 [==============================] - 5s - loss: 0.4160 - acc: 0.8980
Epoch 5/5
8982/8982 [==============================] - 5s - loss: 0.3338 - acc: 0.9136
Test score: 1.01453569163
Test accuracy: 0.797417631398

Finalmente, aquí está mi parte

X_train_sparse = sparse.csr_matrix(X_train)
def batch_generator(X, y, batch_size):
n_batches_for_epoch = X.shape[0]//batch_size
for i in range(n_batches_for_epoch):
index_batch = range(X.shape[0])[batch_size*i:batch_size*(i+1)]
X_batch = X[index_batch,:].todense()
y_batch = y[index_batch,:]
yield(np.array(X_batch),y_batch)
model.fit_generator(generator=batch_generator(X_train_sparse, Y_train, batch_size),
nb_epoch=nb_epoch,
samples_per_epoch=X_train_sparse.shape[0])

El choque:

Exception Traceback (most recent call last)
<ipython-input-120-6722a4f77425> in <module>()
1 model.fit_generator(generator=batch_generator(X_trainSparse, Y_train, batch_size),
2 nb_epoch=nb_epoch,
----> 3 samples_per_epoch=X_trainSparse.shape[0])
/home/kk/miniconda2/envs/tensorflow/lib/python2.7/site-packages/keras/models.pyc in fit_generator(self, generator, samples_per_epoch, nb_epoch, verbose, callbacks, validation_data, nb_val_samples, class_weight, max_q_size, **kwargs)
648 nb_val_samples=nb_val_samples,
649 class_weight=class_weight,
--> 650 max_q_size=max_q_size)
651
652 def evaluate_generator(self, generator, val_samples, max_q_size=10, **kwargs):
/home/kk/miniconda2/envs/tensorflow/lib/python2.7/site-packages/keras/engine/training.pyc in fit_generator(self, generator, samples_per_epoch, nb_epoch, verbose, callbacks, validation_data, nb_val_samples, class_weight, max_q_size)
1356 raise Exception(''output of generator should be a tuple ''
1357 ''(x, y, sample_weight) ''
-> 1358 ''or (x, y). Found: '' + str(generator_output))
1359 if len(generator_output) == 2:
1360 x, y = generator_output
Exception: output of generator should be a tuple (x, y, sample_weight) or (x, y). Found: None

Creo que el problema se debe a una configuración incorrecta de samples_per_epoch. Realmente apreciaría si alguien pudiera comentar sobre esto.

Answer 1

@ BigBoy1337, aquí está mi solución.

def batch_generator(X, y, batch_size):
number_of_batches = samples_per_epoch/batch_size
counter=0
shuffle_index = np.arange(np.shape(y)[0])
np.random.shuffle(shuffle_index)
X = X[shuffle_index, :]
y = y[shuffle_index]
while 1:
index_batch = shuffle_index[batch_size*counter:batch_size*(counter+1)]
X_batch = X[index_batch,:].todense()
y_batch = y[index_batch]
counter += 1
yield(np.array(X_batch),y_batch)
if (counter < number_of_batches):
np.random.shuffle(shuffle_index)
counter=0

En mi caso, X - matriz dispersa, y - matriz.

Answer 2

Si puedes usar lasaña en lugar de Keras, he escrito una pequeña clase de MLP con las siguientes características:

admite matrices densas y dispersas

admite el abandono y la capa oculta

Admite la distribución de probabilidad completa en lugar de etiquetas únicas para admitir el entrenamiento multilable.

Compatible con scikit-learn como API (ajuste, predicción, precisión, etc.)

Es muy fácil de configurar y modificar