For this first experiment I am going to start with a simple goal, getting a neural network to forget one output class, while maintaining as much performance with the other classes as possible. The model is a fully connected neural network trained on MNIST. The method I am going to use is called mean activated absolute value difference (MAAV difference). When a datapoint is passed through the model, each weight is multiplied by an input, this is its activated value. To get the MAAV get the absolute value of this, then the mean of them from a batch of data. Getting the MAAV difference involves seperating a target class from the training data, and passing through the main training data, and the target class data, and finding the MAAVs and then the difference between the 2. The weights with the highest MAAV difference will have a higher average value when the target class is passed through than when the main dataset is passed through. The weights with the highest value should be more closely associated with the target class.
I had to build a couple of things for this experiment. FIrst thing was to modify the data loader to seperate out the target class from the main dataser. Then I had to modify the module I use to prune models, called the lobotomizer. I previously passed one batch of data through the model to find the MAAVs, now I need 2. Then I find the difference. If you are interested in the code, then I cover this in more detail in the notebook.
The model that I am using is a 4 layer neural net with layer sizes of 1000, 800, 400 activations, and an output layer with 10 classes. I am going to see what happens when I try and remove parameters from the middle 2 layers. I will discuss the reasons for not pruning the first and last layers in the next blog.
Well that went better than I was expecting. Almost too well I think. By the time we are removing 6500 – 7000 parameters from each layer, the accuracy of the target data is no better than random, and main data accuracy is unchanged. IF that is all there was to it I would call this a great success. The model has forgotten what a 7 is, and remembers everything else. This is from 2 layers, so 12-13,000 parameters which is quite a small ammount, success there too. There is a significant effect while the number is still in the hundreds, which is very few parameters, and they do indeed change the behaviour of the model in the intended way. But that is not all there is to it, so here is what happens when these layers are pruned individually.
The axes are different here. Many more weights are being removed, 20% of the biggest layer. Target accuracy drops much slower, and the main accuracy starts to drop slowly when the target accuracy is still at about 60%.
Here the target accuracy only drops to 40%, even when 50% of the total parameters from this layer are removed. The main accuracy does not drop by very much at all. One thing that we can understand from this is that most of the parameters being removed here are not that important to the overall functioning of the model on their own. I stopped at 160,000 parameters because it shows what I need to here.
When I pruned both layers, I thought that this was working too well, something must be wrong. This could not be that easy, this is when I decided to prune the individual layers. I was expecting the 3rd layer (bottom plot) to be much more effective, but was wrong there. Pruning the second was more effective, but clearly there is something about pruning both that is far more effective. More than the sum of their parts.
I did prune both layers further than the 7,000 parameters I show in the first plot here, and the target accuracy drops below giving a random output. The main accuracy also starts to drop at that point. I will discuss this in the next blog. As for the other layers, this was enough pruning for this experiment. Removing more than 50% of the parameters of a layer is out of the scope of this experiment, and is the sort of thing that gets me bogged down for weeks looking for things that are not relevant.
I was going to look at a couple of other classes. This is for 7s. I expected them to be pretty similar, but after a quick look at trying to forget 8s, it seems different enough to warrant a bit more than a passing comment here. Classes that seem similar enough might be easier or harder to forget. This will be simple enough for a blog similar to this one, so maybe in 3 blogs time.
Next
Next is just a bit more detail and discussion of what I have done here. After that is another experiment… not really an experiment, I am just going to look at the ourput values, and activations to try and understand a bit better what is going on. I am going to try a little bit to keep blogs with results seperate from ones where I am heavy on the spectulation.