Fuse

Attention Fusion

class imml.fuse.AttentionFusion(n_features: int, bias=False)

Bases: object

PyTorch module to fuse modalities using the attention mechanism. [1]

References

[1]

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. Advances in neural information processing systems. 2017;30.

Example

>>> import numpy as np
>>> import pandas as pd
>>> from imml.fuse import AttentionFusion
>>> Xs = [torch.from_numpy(np.random.default_rng(42).random((20, 10))) for i in range(3)]
>>> fuse = AttentionFusion()
>>> fuse(Xs)

Concat Fusion

class imml.fuse.ConcatFusion

Bases: object

PyTorch module to fuse modalities by concatenating them.

Example

>>> import numpy as np
>>> import pandas as pd
>>> from imml.fuse import ConcatFusion
>>> Xs = [torch.from_numpy(np.random.default_rng(42).random((20, 10))) for i in range(3)]
>>> fuse = ConcatFusion()
>>> fuse(Xs)

EmbraceNet

class imml.fuse.EmbraceNet(missing_values: float = None)

Bases: object

PyTorch module to fuse modalities using EmbraceNet. [2] [3] [4]

Parameters:

missing_values (float, default=0.) -- Value to use for missing data.

References

[2]

Choi JH, Lee JS. EmbraceNet: A robust deep learning architecture for multimodal classification. Information Fusion. 2019 Nov 1;51:259-70.

[3]

https://github.com/luisvalesilva/multisurv/tree/master

[4]

https://github.com/idearibosome/embracenet

Example

>>> import numpy as np
>>> import pandas as pd
>>> from imml.fuse import EmbraceNet
>>> Xs = [torch.from_numpy(np.random.default_rng(42).random((20, 10))) for i in range(3)]
>>> fuse = EmbraceNet()
>>> fuse(Xs)

Max Fusion

class imml.fuse.MaxFusion

Bases: object

PyTorch module to fuse modalities using the max operation.

Example

>>> import numpy as np
>>> import pandas as pd
>>> from imml.fuse import MaxFusion
>>> Xs = [torch.from_numpy(np.random.default_rng(42).random((20, 10))) for i in range(3)]
>>> fuse = MaxFusion()
>>> fuse(Xs)

Mean Fusion

class imml.fuse.MeanFusion

Bases: object

PyTorch module to fuse modalities using the mean operation.

Example

>>> import numpy as np
>>> import pandas as pd
>>> from imml.fuse import MeanFusion
>>> Xs = [torch.from_numpy(np.random.default_rng(42).random((20, 10))) for i in range(3)]
>>> fuse = MeanFusion()
>>> fuse(Xs)

Sum Fusion

class imml.fuse.SumFusion

Bases: object

PyTorch module to fuse modalities using the sum operation.

Example

>>> import numpy as np
>>> import pandas as pd
>>> from imml.fuse import SumFusion
>>> Xs = [torch.from_numpy(np.random.default_rng(42).random((20, 10))) for i in range(3)]
>>> fuse = SumFusion()
>>> fuse(Xs)