Info
Partners
Dictionary Learning on Manifolds
International Workshop
International Workshop
In the domain of signal and image processing, finding and using appropriate representations for the data being analysed is a critical step in extracting valuable information. Recent advances in methodology and optimisation investigate the use of sophisticated learning and modelling strategies to preserve the intrinsic properties of physical signals that typically live in low-dimensional non-linear manifolds. Potential applications include astronomy, astrophysics, medicine and computer vision.
The purpose of this international workshop is to discuss cutting-edge ideas and explore new strategies to model and restore signals living in these known or unknown complicated spaces. Topics include dictionary learning on manifolds, deep learning, optimisation and inverse problems on manifolds.
Jean-Luc Stark
Panagiotis Tsakalides
Filipe Abdalla
Gitta Kutyniok
Marc Bousquet
Jean-Luc Starck
Florent Sureau
Samuel Farrens
Speaker: Loubna El Gueddari
Abstract: Magnetic Resonance Imaging (MRI) is a widely adopted imaging technique for clinical diagnosis due to its non invasiveness. However, the main limitation of MRI is its long acquisition time. Compressed Sensing (CS) provides us with a way to reduce this time by using under-sampled non-Cartesian sampling schemes, but it may require good SNR conditions. To ensure good SNR conditions, one can take advantage of the Parallel Imaging (PI) technique, where multiple coils are used to collect the signal. However, the combination of PI and CS for non-Cartesian patterns may suffer from long iterative reconstruction process. To overcome this issue, algorithms which tend to reduce both the number of iterations to convergence and the computation time per iteration are preferred. In this work, we use a Proximal Optimized Gradient Method (POGM) combined with a sensitivity coil based majorization and we illustrate its efficiency with a prospective acquisition at 7T scanner accelerated by a factor of 15.
Slides: PDF
Speaker: Carola-Bibiane Schönlieb
Abstract: One of the most successful approaches to solve inverse problems in imaging is to cast the problem as a variational model. The key to the success of the variational approach is to define the variational energy such that its minimiser reflects the structural properties of the imaging problem in terms of regularisation and data consistency.
Variational models constitute mathematically rigorous inversion models with stability and approximation guarantees as well as a control on qualitative and physical properties of the solution. On the negative side, these methods are rigid in a sense that they can be adapted to data only to a certain extent.
Hence researchers started to apply machine learning techniques to “learn” more expressible variational models. In this talk we discuss two approaches: bilevel optimisation (which we investigated over the last couple of years and which aims to find an optimal model by learning from a set of desirable training examples) and quotient minimisation (which we only recently proposed as a way to incorporate negative examples in regularisation learning). Depending on time, we will review the analysis of these approaches, their numerical treatment, and show applications to learning sparse transforms, regularisation learning, learning of noise models and of sampling patterns in MRI.
Slides: PDF
Speaker: Grigorios Tsagkatakis and Joana Frontera-Pons
Abstract: The interest generated by the so-called Deep Learning methods has thrived during the last decade. The success encountered by such methods is primarily attributed to the wealth of available labelled data, including measurements from modern instruments, in conjunction with the dramatic increase in computational performance. Indeed, they hold the state of the art results in classical problems such as computer vision, speech recognition and natural language processing, among others. In this talk, we will first provide a quick introduction and historical overview of deep learning. Subsequently, we will expand on the latest research in unsupervised and supervised learning, with focus in algorithms like autoencoders and convolutional neural networks. Applications of these paradigms will be focused on problems from Astrophysics and will include live demos.
Tutorial Materials:
Speaker: George Tzagkarakis
Abstract: Financial markets are complex systems whose underlying dynamics are governed by a manifold structure. In this work we study a method for financial markets integration based on manifold learning. First, we restructure the phase space of a financial dynamic system using the associated financial time series data. Then, we exploit an information metric-based manifold learning approach to extract the intrinsic manifold structure of the system. Given the underlying learnt manifold, a framework is proposed for identifying markets integration being able to capture the critical transitions of financial markets. We deduce the intrinsic geometric properties of the manifold to react against impending crises. Experimental results reveal that the proposed approach describes accurately the inherent manifold-like structure of the financial dynamic system, and contributes to inform investors about the state of financial markets.
Slides: PDF
Speaker: Philippe Ciuciu
Abstract: Magnetic Resonance Imaging (MRI) is one of the most powerful non-invasive imaging techniques used in clinical routine today. However, its lengthy scan times still prevent realizing its full potential especially for high resolution imaging. MR sampling schemes used to collect the Fourier data along continuous trajectories have remained essentially unchanged for the last 30 years. To speed up acquisitions, under-sampled non-Cartesian patterns are usually considered. Recently, we have introduced a mathematically principled approach that determines time-optimal feasible sampling trajectories, called SPARKLING (SPARse K-space sampLING). When combined with Compressed Sensing (CS) and parallel imaging (PI), the proposed strategy permits to reach unprecedented acceleration factors (e.g, x15) in 2D while mitigating under-sampling and motion artifacts as well as signal-to-noise ratio concerns. Prospective acquisitions performed in vivo on a 7 Tesla scanner proved the superiority of this approach over both radial and spiral readouts in terms of image quality after nonlinear reconstruction in the non-Cartesian CS-PI setting. To accelerate the convergence speed of the CS-PI reconstruction, we used a proximal optimal gradient method (POGM) combined with a matrix-valued majorization based on coil sensitivities, also called BARISTA in the literature.
Slides: PDF
Speaker: Matthieu Heitz
Abstract: This article introduces a new dictionary learning method for histograms in the probability simplex. That method leverages optimal transport theory, in the sense that our aim is to reconstruct datapoints using so called displacement interpolations (a.k.a Wasserstein barycenters) of a few atoms. Our method simultaneously estimates shared atoms across all datapoints, and, for each datapoint, the set of weights that can optimally reconstruct it as an optimal transport barycenter of such atoms. Our method is computationally tractable thanks to the addition of an entropic regularization to the usual optimal transportation problem, leading to an efficient approximation scheme that is efficient, parallel and simple to differentiate. Both atoms and weights are learned by gradient descent; gradients are computed using automatic differentiation schemes of generalized Sinkhorn iterations used to compute barycenters with entropic smoothing. Because of its formulation relying on Wasserstein barycenters instead of the usual matrix product between dictionary and codes, our method allows for non-linear relationships between atoms and the reconstruction of input data. We illustrate its application in several different image processing settings.
Slides: PDF
Speaker: Christophe Kervazo
Abstract: Standard Blind Source Separation (BSS) methods allow to efficiently analyze large-scale multichannel data but they fail at dealing with problems involving a large number of sources. Building upon standard BSS algorithms, we propose investigating the performances of block-coordinate optimization strategies to extend the framework of BSS to high numbers of sources. The use of blocks of intermediate sizes will be shown to allow both an improvement in the separation quality and a gain in computation time.
Slides: PDF
Speaker: Melis Irfan
Abstract: Within cosmology component separation is instrumental in disentangling information pertaining to the formation and evolution of the Universe itself from astrophysical information describing the Universe’s key constituents. Separating distinct, non-linear emission mechanisms within large-scale data sets is a complex problem which can benefit from a prior knowledge of said emission mechanisms. We present a new model fitting technique which exploits the sparse nature of these emissions to provide more accurate model parameters in less computational time than the current state-of-the-art methodologies. This technique automatically identifies regions within the data with common properties, makes fast, initial parameter estimates and then refines these regional estimates to provide the global solution through a least squares optimisation which favours sparsity. We verify our method in the context of Planck simulation data.
Slides: PDF
Speaker: Niall Jeffrey
Abstract: Dark matter mass maps from weak lensing are rich in information about cosmological parameters and models of the universe. With accurate map reconstructions using GLIMPSE and realistic simulations, we can extract extra information from the maps, beyond what is achievable by traditional methods. In this presentation I will show my work to extract this information using peak statistics.
Slides: PDF
Speaker: Kostas Themelis
Abstract: In this turorial we explain the foundations of Approximate Bayesian Computation (ABC), a method that performs Bayesian inference without the need to explicitly define a likelihood function. Instead, ABC only requires that sampling from a model is possible. Hence, ABC can be utilized to estimate posterior distributions of parameters in simulation-based models. We present the basic algorithms for ABC that are based on the principle of rejection sampling and we use Python code to illustrate the effectiveness of ABC in several examples.
Tutorial Materials:
Speaker: Domenico Marinucci
Abstract: Abstract: We present a topological multiple testing scheme for detecting
peaks on the sphere under isotropic Gaussian noise, where tests are
performed at local maxima of the observed field filtered by the spherical
needlet transform. Our setting is different from the standard
Euclidean/large same asymptotic framework, yet highly relevant to
realistic experimental circumstances for some important areas of
application in astronomy. More precisely, we focus on cases where a single
realization of a smooth isotropic Gaussian random field on the sphere is
observed, and a number of well-localized signals are superimposed on such
background field. The proposed algorithms, combined with the
Benjamini-Hochberg procedure for thresholding p-values, provide asymptotic
strong control of the False Discovery Rate (FDR) and power consistency as
the signal strength and the frequency of the needlet transform get large.
This novel multiple testing method is illustrated in a simulation of
point-source detection in Cosmic Microwave Background radiation (CMB)
data.
Slides: PDF
Speaker: Adrienne Leonard
Abstract: In this workshop, I will walk through the process of taking an idea, algorithm, or solution from concept through to an industry-ready product. Broadly, this process follows three phases: discovery, prototyping, and productionising. I will highlight the similarities and differences between product/algorithm development in industry and in academia, and showcase some of the standard tools and processes used in industry.
Tutorial Materials:
Speaker: Emmanuel Soubies
Abstract: In this talk, we present a novel statistical dictionary learning algorithm based on an alpha-stable innovation model [1]. The main idea of the method relies on the property that knowing the marginal dispersions of symmetric alpha stable (SaS) random vectors is equivalent to knowing the Radon transform of their PDFs. Hence, considering a signal model y=Ax where x is an unknown SaS random vector, A is a dictionary matrix and y is an observation vector, it turns out that A is identifiable from an appropriate set of data projections. The identification is performed by minimizing a new cost function by means of a gradient based algorithm. Moreover, in order to avoid getting trapped in local minima, the set of data projections is randomly changed during algorithm iterations. This allows to modify the nonconvex objective and its local minima, without changing global ones, resulting in an efficient and robust algorithm.
Results obtained in the context of 3D deconvolution widefield microscopy will be presented with comparisons to other competing methods. Moreover, in order to enhance axial resolution of the deconvolved images, we employ an original strategy that consists in learning a dictionary from the highly resolved lateral sections (x,y) and apply it in both lateral and axial directions.
[1] Pedram Pad, Farnood Salehi, Elisa Celis, Patrick Thiran and Michael Unser. Dictionary Learning Based on Sparse Distribution Tomography. Proceedings of the 34th International Conference on Machine Learning 2017.
Slides: PDF
Speaker: Valeryia Naumova
Abstract: Technological advances, such as in physical measurements, computer simulations, and storage capabilities led to vast amounts of often highly complex data sets, and this continues to grow rapidly. Despite substantial R&D to develop tools for complex data analysis, in many cases our understanding on how to extract useful information and predictive models still remains rather limited.
Motivated by the increased demand of robust predictive methods, we present an approach to explore new applications in tractable and robust automatic learning of functions and data structures in high dimension from the minimal number of observed samples. Our approach aims at obtaining lower-dimensional representations of the data and their geometry with significantly reduced complexity, assuming the data clustered around manifolds. The key innovative assumption for us is that the underlying manifold, not only possesses lower dimension, but its tangent spaces are also spanned by relatively sparse principal directions. Additionally we consider the learning of the manifold as guided by the function acting on the data. Hence, our approach differs from most of the established methods on manifold learning and establishes a novel connection between manifold and function learning in high dimension.
Slides: PDF