Main goal of this project is to provide trainable representations of real-word input data to regular neural networks.
There are two classes implemented so far: FuzzyLayer
and DefuzzyLayer
.
This layer is suitable for cases when you working with data that can be clustered into interpretable groups e.g. spatial coordinates, multi-function values and etc.
Membership function for this layer have form:
where x
is an input vector of dim
length, c
is centroid of j-th membership function and a
is vector of scaling factors.
DefuzzyLayer
can be trained to transform output of an model to continuos values. In other words this layer can be interpreted as an ruleset and input to this layer - firing levels for rules.
Membership function for layer FuzzyLayer2
have form
with
Main benefit of FuzzyLayer2
over FuzzyLayer
is that fuzzy centroids are aligned in arbitrary direction to cover cluster structures more preciesly.
import keras
from FuzzyLayer import FuzzyLayer
from DefuzzyLayer import DefuzzyLayer
from keras.models import Sequential
...
model = Sequential()
model.add(FuzzyLayer(20, input_dim=2))
model.add(DefuzzyLayer(1))
model.compile(loss='logcosh',
optimizer='rmsprop',
metrics=['mae'])
model.fit(x_train, y_train,
epochs=500,
verbose=1,
batch_size=100)
latent_dim = 3
mnist_inputs = keras.Input(shape=(28, 28, 1))
x = layers.Conv2D(64, 3, activation="relu", padding="same")(mnist_inputs)
x = layers.Conv2D(64, 3, activation="relu", padding="same")(x)
x = layers.Conv2D(32, 5, activation="relu", strides=2, padding="same")(x)
shape_before_flattening = K.int_shape(x)
x = layers.Flatten()(x)
z = layers.Dense(latent_dim, name="z")(x)
base_model = keras.Model(mnist_inputs, z)
x = base_model(mnist_inputs)
x = FuzzyLayer2(10, name="fuzzy")(x)
x = DefuzzyLayer(10, name="defuzzy")(x)
x = tf.keras.layers.Softmax()(x)
fuzzy_model = keras.Model(mnist_inputs, x)
Accuracy achieved by training fuzzy_model
is about 0.995 and presented model has nice clustered latent layer z
structure:
MNIST fuzzy anomaly detection with VAE
MNIST fuzzy anomaly detection with CVAE
MNIST semi-supervised (partially labeled dataset) learning with fuzzy CVAE