Projects

Intrinsic Image Transfer for Illumination Manipulation

We presetnt a novel intrinsic image transfer (IIT) algorithm for image illumination manipulation, which creates a local image translation between two illumination surfaces. This model is built on an optimization-based framework composed of illumination, reflectance and content photo-realistic losses, respectively. Each loss is firstly defined on the corresponding sub-layers factorized by an intrinsic image decomposition and then reduced under the well-known spatial-varying illumination illumination-invariant reflectance prior knowledge. We illustrate that all losses, with the aid of an exemplar image, can be directly defined on images without the necessity of taking an intrinsic image decomposition, thereby giving a closed-form solution to image illumination manipulation. We also demonstrate its versatility and benefits to several illumination-related tasks &comma illumination compensation, image enhancement and tone mapping, and high dynamic range (HDR) image compression, and show their high-quality results on natural image datasets.

Optimal Image Transport on Sparse Dictionaries

We derive a novel optimal image transport algorithm over sparse dictionaries by taking advantage of Sparse Representation (SR) and Optimal Transport (OT). Concisely, we design a unified optimization framework in which the individual image features (color, textures, styles, etc.) are encoded using sparse representation compactly, and an optimal transport plan is then inferred between two learned dictionaries in accordance with the encoding process. This paradigm gives rise to a simple but effective way for simultaneous image representation and transformation, which is also empirically solvable because of the moderate size of sparse coding and optimal transport sub-problems. We demonstrate its versatility and many benefits to different image-to-image translation tasks, in particular image color transform and artistic style transfer, and show the plausible results for photo-realistic transferred effects.

Semi-sparsity for Smoothing Filters

We propose a semi-sparsity smoothing method based on a new sparsity-induced minimization scheme. The model is derived from the observations that semi-sparsity prior knowledge is universally applicable in situations where sparsity is not fully admitted such as in the polynomial-smoothing surfaces. We illustrate that such priors can be identified into a generalized $L_0$-norm minimization problem in higher-order gradient domains, giving rise to a new 'feature-aware' filter with a powerful simultaneous-fitting ability in both sparse singularities (corners and salient edges) and polynomial-smoothing surfaces. Notice that a direct solver to the proposed model is not available due to the non-convexity and combinatorial nature of $L_0$-norm minimization. Instead, we propose to solve it approximately based on an efficient half-quadratic splitting technique. We demonstrate its versatility and many benefits to a series of signal/image processing and computer vision applications.

Semi-sparsity Priors for Image Structure Analysis and Extraction

Image structure-texture decomposition is a long-standing and fundamental problem in both image processing and computer vision fields. In this paper, we propose a generalized semi-sparse regularization framework for image structural analysis and extraction, which allows us to decouple the underlying image structures from complicated textural backgrounds. Combining with different textural analysis models, such a regularization receives favorable properties differing from many traditional methods. We demonstrate that it is not only capable of preserving image structures without introducing notorious staircase artifacts in polynomial-smoothing surfaces but is also applicable for decomposing image textures with strong oscillatory patterns. Moreover, we also introduce an efficient numerical solution based on an alternating direction method of multipliers (ADMM) algorithm, which gives rise to a simple and maneuverable way for image structure-texture decomposition. The versatility of the proposed method is finally verified by a series of experimental results with the capability of producing comparable or superior image decomposition results against cutting-edge methods.

Semi-sparsity on Piecewise Constant Function Spaces for Triangular Mesh Denoising

We present a semi-sparsity model for 3D triangular mesh denoising, which is motivated by the success of semi-sparsity regularization in image processing applications. We demonstrate that such a regularization model can be also applied for graphic processing and gives rise to the similar simultaneous-fitting results in preserving sharp features and piece-wise smoothing surfaces. Specifically, we first describe the piecewise constant function spaces associated with the differential operators on triangular meshes and then show how to extend the semi-sparsity model to meshes denoising. To verify its effectiveness, we present an efficient iterative algorithm based on alternating direction method of multipliers (ADMM) technique and show the experimental results on synthetic and real scanning data against the state-of-the-arts both visually and quantitatively.

(Semi-) Sparsity Prior Knowledge for Image/Signal Processing

The project aims to using (semi-)sparsity prior knowledge for a wide range of image/signal processing. Specifically, we have demostrated that (semi-)sparsity prior knowledge can be formulated into a class of generalized higher-order variation methods, which are beneficial for a wide range of image processing and computer vision tasks, such as image/signal filtering, inverse imaging, image structure-texture decomposition, depth/stereo estimation, and so on.

Deep Low-level Vision Learning towards Practical Applications

Design a real-time image rendering framework to improve image clarity and contrast automatically with little artifacts. The project involves several low-level image processing tasks, including image demosaicking, denoising, tone mapping, enhancement, dehaze and so on. It also involves several theoretical research fields such as wavelet analysis, filtering techniques, deep learning and so on.

Artistic Stylization Algorithm for Images and Videos

Develop (deep) learning-based methods for image and video artistic stylization, style transfer, and non-photorealistic image rendering, etc..

Real-time Image/Video Rendering for High-quality Perception

Design a real-time image rendering framework to improve image clarity and contrast automatically with little artifacts. The project involves several low-level image processing tasks, including image demosaicking, denoising, tone mapping, enhancement, dehaze and so on. It also involves several theoretical research fields such as wavelet analysis, filtering techniques, deep learning and so on.

Depth (Steoro, Optical Flow) Estimation, 3D Reconstruction and Robot SLAM

The project mainly focuses on depth (steoro, optical flow) estimation for high-resolution images, which aims to give more accurate results in salient edge for the edge-aware preserving property. A real-time SLAM system and 3d dense reconstruction, following the start-of arts like ORB SLAM, LSD SLAM, SVO SLAM, and so on.