# A Note on Layering Classifications

In a previous research note, I showed how to use my deep learning application Prometheus to do video classifications. The technique I employed was to apply image classification to each frame of the video, which produces a vector of classifications for each video file. That is, a video consists of some number of frames, and so if we classify each frame as an image, we will generate a vector of classifications that contains as many entries as the video contains frames.

I’ve been experimenting with broadening this approach, by repeatedly applying my classification and prediction algorithm. Specifically, I build a first model using the “generate data tree” function, as applied to a portion of the training set. Then, I take a new portion of the training set, and feed it to the prediction algorithm, which will generate classification predictions. However, during this process, it compares each input to every entry in the data tree that underlies the prediction model. By saving saving this information in a vector, we’ll generate a vector of differences associated with each input.

We can then classify those vectors of differences, by calling the “generate data tree function” again, using those vectors as the training dataset. If we do this repeatedly, we’re going to generate a chain of models, and we can then pump new testing inputs through that chain, generating a classification prediction at each stage of the chain. The end result of this process will be a vector of classifications: the same product produced by my video classification technique. We can then classify those vectors of classifications, with the expectation that this final classification will make use of multiple layers of information extracted from the original dataset.

We could also generate a chain of length three, and then put the bottom two models in the chain into a loop. That is, if the models are M1, M2, and M3, we could run the output of M3 back into the input of M2, producing a loop that would run some fixed number of times, generating a fixed number of classifications. This would cut down on model generation time, but still allow us to generate a vector of classifications. You need a minimum of three models to do this because the input to M1 is the actual training data, whereas the inputs to M2 and M3 are vectors of differences.

I’m also going to experiment with random “filters”, rather than using the trees generated during this process. That is, we compare the vectors to randomly generated vectors, measure the differences, and store them as a difference vector. The hope would be that each class will have a signature pattern when applied to the same set of filters.

I’m currently testing this and will follow up in the next few days with code and a research note.