All posts by comitchell

Kernel-based generalized median computation for consensus learning

Andreas Nienkötter received a master’s degree in computer science from University of Münster, Germany, in 2015, followed by a Ph.D. in 2021. He is currently in a Postdoctoral position in Prof. X. Jiang’s Research Group for Pattern Recognition and Image Analysis. His research interests include consensus learning using the generalized median, vector space embedding methods, and dimensionality reduction methods.

ULTRACEPT researcher Andreas Nienkötter from the University of Münster, Germany recently published a journal article titled ‘Kernel-based generalized median computation for consensus learning’ in IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, doi: 10.1109/TPAMI.2022.3202565. The journal covers research in computer vision and image understanding, pattern analysis and recognition, machine intelligence, machine learning, search techniques, document and handwriting analysis, medical image analysis, video and image sequence analysis, content-based retrieval of image and video, and face and gesture recognition.

Research Summary

Consensus learning is an important field in computer science with application in many diverse fields. The task is to compute a single representative of a given set of singular data or results, e.g. a final prediction from a set of input predictions, an averaged rotation or position from several measured rotations or positions, or a representative DNA-sequence computed from several faulty or mutated DNA sequences.

The generalized median is a very intuitive method to compute such a consensus related to the standard median of numbers. The median of numbers is not only the value in the centre of the sorted list, but also a value that minimizes the sum of absolute differences to the values in the list.

Transferred to general domains such as positional data, genetic strings, rotation matrices, physical signals, the generalized median is an object which minimizes the sum of distance to the input set given a domain specific distance. Surprisingly, the computation of the generalized median is often NP-hard or even worse, even for simple objects such as two-dimensional vectors with the euclidean distance. Nevertheless, the generalized median is popular in practice as it is much more robust against outliers compared to simple averaging.

In this research, we developed a method to compute the generalized median in any domain requiring only minimal user input by computing the median in kernel space. The system is based on the distance-preserving embedding method used in our previous work, as seen in Figure 1.

Overview of the distance-preserving embedding method
Figure 1: Overview of the distance-preserving embedding method

The distance preserving embedding framework computes the generalized median in three steps. 1) The objects are embedded into a high-dimensional vector space so that the pairwise distances between vectors are approximately the same as the pairwise distances between the input objects. 2) The generalized median of these vectors (often called geometric median) is approximated in vector space, for which fast algorithms exist. 3) Finally, the median is reconstructed by combining the objects corresponding to the neighbours of the median in vector space.

Research Highlights

While this previous framework has been shown to accurately compute a generalized median, it has a major drawback: the embedding into an euclidean vector space. Unfortunately, most distances (e.g. the string edit distance) cannot be accurately represented in an euclidean vector space, leading to distortions in the position of vectors and ultimately to a less accurate median computation. To overcome this, the kernel-based median computation in this work removes the need for an explicit vector space entirely by using kernel functions instead.

As we were able to show in our work, the above framework does not actually need the explicit vectors of objects or even the explicit vector position of the generalized median. The scalar product between these vectors suffices instead. These scalar products can be efficiently computed using kernel functions. Kernel functions are defined as functions computing the scalar product of objects after transformation into a Hilbert space. Notably, this space can be unknown and even infinite-dimensional, something that was not possible before with explicit embedding methods. Some of these kernel functions only require the distances between objects, meaning that there is no additional input required by the user. These easy to use kernel functions even compute the scalar product in a pseudo-euclidean space leading to a more accurate representation of distances, something that was not possible with an euclidean vector space before.

As seen in Figure 2, we were able to achieve superior results using this method compared to previous methods in string, clustering and ranking data. The results were computed with less parameters in a similar time.

Average rank of different median algorithms on string, clustering and ranking datasets. Lower is better. The Kδ are our method using different kernel functions. CCA and Prototype are previous methods with explicit embedding.
Figure 2: Average rank of different median algorithms on string, clustering and ranking datasets. Lower is better. The Kδ are our method using different kernel functions. CCA and Prototype are previous methods with explicit embedding.

Generalized Median Toolbox

The kernel-based generalized median method as well as the previous distance preserving embedding methods are available to the public as python toolbox (https://pypi.org/project/gmtb/). We invite anyone interested in this topic to try it in their consensus computation tasks.

ULTRACEPT Sandpit: Looming Detection: A unique instance of motion detection, from Neural Computing to Drone Applications

To aid and support the continued collaboration and knowledge exchange of the ULTRACEPT researchers, the consortium hosts online quarterly ‘Sandpit Sessions’. The aim of these sessions is to provide researchers an opportunity to share their work in an informal forum where they can raise and discuss issues and challenges in order to gain support and feedback from the group.

Researcher Dr Jiannan Zhao presented at an ULTRACEPT Sandpit Session on the 26th September, 2022. Dr Zhao undertook his PhD at the University of Lincoln, during which, he carried out a 12 month ULTRACEPT secondment to Guangzhou University (GZHU). Dr Zhao is now working at Guangxi University, China.

ULTRACEPT Sandpit 6

The theme of the sandpit session was Looming Detection: A unique instance of motion detection, from Neural Computing to Drone Applications. 24 attendees across the ULTRACEPT consortium participated.

ULTRACEPT Sandpit 6

ULTRACEPT: Ultra-layered perception with brain-inspired information processing for vehicle collision avoidance

Looming Detection: A unique instance of motion detection, from Neural Computing to Drone Applications

  • Date: Monday, 26th September 2022
  • Time: UK 11:00; China 18:00; Germany 12:00; Argentina 07:00; Malaysia 18:00; Japan 19:00.
  • Facilitators:
    • Jiannan Zhao, Presenter, Guangxi University, CN
    • Shigang Yue, University of Lincoln, UK (chair)
Sandpit Schedule
UK Time Item Presenter/s
11:00-11:05 Arrival and welcome Shigang
11:05-12:05 Looming Detection: A unique instance of motion detection, from Neural Computing to Drone Applications Jiannan
12:05-12:35 Group discussion All
12:35-12:45 Final comments & volunteer for a facilitator for the next session. Shigang

ULTRACEPT Sandpit 6

Yicheng Zhang Presents at ICANN 2022

Artificial Neural Networks and Machine Learning (ICANN) 2022 is the 31st International Conference on Artificial Neural Networks. The conference was organised by the Department of Computer Science and Creative Technologies of the University of the West of England at Frenchay Campus, Bristol, from 6 to 9 September 2022. It was held in a hybrid mode with delegates attending on-site and remotely via an immersive online space.

ICAN2022

This conference featured two main tracks: Brain inspired computing and Machine learning research, with strong cross-disciplinary interactions and applications. The event attracted a large number and wide range of new and established researchers from five continents and 27 countries in total. The research themes explored all innovative pathways in the wider area of Neural Networks and Machine Learning. 561 papers were submitted with 259 selected to be presented orally at the conference.

ICAN2022 ICAN2022

ULTRACEPT researcher Yicheng Zhang presented his research Zhang, Y. et al. (2022). O-LGMD: An Opponent Colour LGMD-Based Model for Collision Detection with Thermal Images at Night. In: Pimenidis, E., Angelov, P., Jayne, C., Papaleonidas, A., Aydin, M. (eds) Artificial Neural Networks and Machine Learning – ICANN 2022. ICANN 2022. Lecture Notes in Computer Science, vol 13531. Springer, Cham. https://doi.org/10.1007/978-3-031-15934-3_21. This was presented in session 41 of the conference.

O-LGMD: An Opponent Colour LGMD-Based Model for Collision Detection with Thermal Images at Night

O-LGMD: An Opponent Colour LGMD-Based Model for Collision Detection with Thermal Images at Night

Abstract

It is an enormous challenge for intelligent robots or vehicles to detect and avoid collisions at night because of poor lighting conditions. Thermal cameras capture night scenes with temperature maps, often showing different pseudo-colour modes to enhance the visual effects for the human eyes. Since the features of approaching objects could have been well enhanced in the pseudo-colour outputs of a thermal camera, it is likely that colour cues could help the Lobula Giant Motion Detector (LGMD) to pick up the collision cues effectively. However, there is no investigation published on this aspect and it is not clear whether LGMD-like neural networks can take pseudo-colour information as input for collision detection in extreme dim conditions. In this study, we investigate a few thermal pseudo-colour modes and propose to extract colour cues with a triple-channel LGMD-based neural network to directly process the pseudo-colour images. The proposed model consists of three sub-networks, each dealing with one specific opponent colour channel, i.e. black-white, red-green, or yellow-blue. A collision alarm is triggered if any channel’s output exceeds its threshold for a few successive frames. Our experiments demonstrate that the proposed bio-inspired collision detection system works well in quickly detecting colliding objects in direct collision course in extremely low lighting conditions. The proposed method showed its potential to be part of sensor systems for future robots or vehicles driving at night or in other extreme lighting conditions to help avoiding fatal collisions.

O-LGMD: An Opponent Colour LGMD-Based Model for Collision Detection with Thermal Images at Night

Yicheng was very grateful for the opportunity to attend this fantastic conference with support from ULTRACEPT.

ULTRACEPT Researchers Present at IEEE WCCI/IJCNN 2022

IEEE WCCI 2022 is the world’s largest technical event on computational intelligence, featuring the three flagship conferences of the IEEE Computational Intelligence Society (CIS) under one roof: the 2022 International Joint Conference on Neural Networks (IJCNN 2022), the 2022 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2022), and the 2022 IEEE Congress on Evolutionary Computation (IEEE CEC 2022). The event was held online and in Padua, Italy.

ULTRACEPT researchers attended the event to share their research:

Shaping the Ultra-Selectivity of a Looming Detection Neural Network from Non-linear Correlation of Radial Motion

University of Lincoln researcher Mu Hua has finished his postgraduate program last July and now is an honorary researcher working on ULTRACEPT’s work package 1. He remotely attended the IEEE World Congress on Computational Intelligence 2022 and orally presented his latest work on lobula plate/lobula columnar type 2(LPLC2) neuropile discovered within neural pathway of fruit flies Drosophila.

Mu Hua presented his recent work on modelling the LPLC2 in his paper titled ‘Shaping the Ultra-Selectivity of a Looming Detection Neural Network from Non-linear Correlation of Radial Motion’.

H. Luan, M. Hua, J. Peng, S. Yue, S. Chen and Q. Fu, “Accelerating Motion Perception Model Mimics the Visual Neuronal Ensemble of Crab,” 2022 International Joint Conference on Neural Networks (IJCNN), Padua, Italy, 2022, pp. 1-8, https://10.1109/IJCNN55064.2022.9892540.

With a pre-recorded video, he orally explained how the proposed LPLC2 neural network realises its ultra-selectivity for stimuli initial location of the whole receptive field, object surface brightness and its preference for approaching motion patterns through high-level non-linear combination of motion clues, demonstrating potential for distinguishing near miss so that true collision can be recognized correctly and efficiently.

 Schematic of visual system of Drosophila. ON channel is represented by solid line while OFF channel by dashed line.
Figure 1. Schematic of visual system of Drosophila. ON channel is represented by solid line while OFF channel by dashed line.

Figure 1 shows the schematic of visual system of Drosophila. ON channel is represented by solid line while OFF channel by dashed line. Blue, red, green and purple areas show different type DSNs. More specifically, the blue one represents neurons interesting in upwards motion whilst the purple one prefers downwards motion; the red line and green shows preference to leftwards and rightwards movement respectively. The dot-headed line shows pathway of visual signals accepted by photoreceptors in retina layer, which are firstly dealt with in lamina, and subsequently separated in medulla into parallel ON/OFF channel with polarised selectivity. After that, signals in ON are passed to various types of DSNs (T4s) in lobula plate layer channel for directional motion calculation.

These visual signals estimated by T4 interneurons are then combined with T5 neurons in OFF channel, and further filtered through to lobula plate tangential cells (LPTCs, represented by orange dots within slightly transparent areas). Note number of lines shall not be the actual number of neurons within Drosophila visual system.

 Illustration of directionally-selective neurons LPTCs being activated by edge expanding (top) and remaining silent against recession (bottom).
Figure 2. Illustration of directionally-selective neurons LPTCs being activated by edge expanding (top) and remaining silent against recession (bottom).

Figure 2 illustrates the directionally-selective neurons LPTCs being activated by edge expanding (top) and remaining silent against recession (bottom). The black circle represents one dark looming motion pattern. As it expands, four sorts of T4/T5 interneurons in four colours sense motions along one of the four cardinal directions. Directional information is then estimated within the T4 or T5 pathway. The ON channel motion estimation in T4 and OFF channel motion in T5 are then summarised by their post-synaptic structure LPTC neurons. The particular placement of LPTC neurons as shown is considered to impose impacts on the following non-linear combining calculation of LPLC2 neurons.

Snapshots of one of the experimental stimuli, where a square lays on a complex background.
Figure 3. Snapshots of one of the experimental stimuli, where a square lays on a complex background.
Snapshots of one of the experimental stimuli, where a square lays on a complex background.
Figure 3. Snapshots of one of the experimental stimuli, where a square lays on a complex background.

Figure 3 shows snapshots of one of the experimental stimuli, where a square lays on a complex background. From top to bottom, the motion pattern is approaching and reversely generates receding. Curves on the right show the output of proposed model and the classic LGMD1 neural network chosen for comparison. The original curve demonstrates that our proposed neural network is only activated by approaching motion pattern, which fits well biological findings.

Abstract

In this paper, a numerical neural network inspired by the lobula plate/lobula columnar type II (LPLC2), the ultraselective looming sensitive neurons identified within the visual system of Drosophila, is proposed utilising non-linear computation. This method aims to be one of the explorations toward solving the collision perception problem resulting from radial motion. Taking inspiration from the distinctive structure and placement of directionally selective neurons (DSNs) named T4/T5 interneurons and their post-synaptic neurons, the motion opponency along four cardinal directions is computed in a non-linear way and subsequently mapped into four quadrants. More precisely, local motion excites adjacent neurons ahead of the ongoing motion, whilst transferring inhibitory signals to presently-excited neurons with slight temporal delay. From comparative experimental results collected, the main contribution is established by sculpting the ultra-selective features of generating a vast majority of responses to dark centroid-emanated centrifugal motion patterns whilst remaining nearly silent to those starting from other quadrants of the receptive field (RF). The proposed method also distinguishes relatively dark approaching objects against the brighter backgrounds and light ones against dark backgrounds via exploiting ON/OFF parallel channels, which well fits the physiological findings. Accordingly, the proposed neural network consolidates the theory of non-linear computation in Drosophila’s visual system, a prominent paradigm for studying biological motion perception. This research also demonstrates the potential of being fused with attention mechanisms towards the utility in devices such as unmanned aerial vehicles (UAVs), protecting them from unexpected and imminent collision by calculating a safer flying pathway.

A Bio-inspired Dark Adaptation Framework for Low-light Image Enhancement

Fang Lei is a PhD Scholar at the University of Lincoln. Fang presented her poster promoting her research ‘A Bio-inspired Dark Adaptation Framework for Low-light Image Enhancement’.

F. Lei, “A Bio-inspired Dark Adaptation Framework for Low-light Image Enhancement,” 2022 International Joint Conference on Neural Networks (IJCNN), Padua, Italy, 2022, pp. 1-8, https://10.1109/IJCNN55064.2022.9892877

Fang Lei's poster at WCCI2022
Fang Lei’s poster at WCCI2022
Abstract

In low light conditions, image enhancement is critical for vision-based artificial systems since details of objects in dark regions are buried. Moreover, enhancing the low-light image without introducing too many irrelevant artifacts is important for visual tasks like motion detection. However, conventional methods always have the risk of “bad” enhancement. Nocturnal insects show remarkable visual abilities at night time, and their adaptations in light responses provide inspiration for low-light image enhancement. In this paper, we aim to adopt the neural mechanism of dark adaptation for adaptively raising intensities whilst preserving the naturalness. We propose a framework for enhancing low-light images by implementing the dark adaptation operation with proper adaptation parameters in R, G and B channels separately. Specifically, the dark adaptation in this paper consists of a series of canonical neural computations, including the power law adaptation, divisive normalization and adaptive rescaling operations. Experiments show that the proposed bioinspired dark adaptation framework is more efficient and can better preserve the naturalness of the image compared to existing methods.

Model

The proposed bio-inspired dark adaptation framework is shown in Fig.1. The key idea of the dark adaptation is to adaptively raise the intensities of dark pixels by a series of canonical neural computations (see Fig.2).

Proposed dark adaptation framework for low light image enhancement
Figure 1. Proposed dark adaptation framework for low light image enhancement. The red (R), green (G), and blue (B) components of the input image are processed with the dark adaptation in three separate channels. Note that each channel has a different adaptation parameter.
Schematic illustration of the proposed dark adaptation
Figure 2. Schematic illustration of the proposed dark adaptation. There are N cells that correspond to N pixels in the input image, denoted by I1 ∼ IN. n denotes the sensation parameter, and its value depends on the wavelength of perceived light. Ii and I′i indicate the ith cell and its enhanced output after the dark adaptation processing. For clear illustration, we only give one cell’s enhanced result.

Results

We compare the proposed method with existing low light image enhancement methods, including comparisons of visual performance, lightness order error (LOE), and average running time. The experimental results are shown below.

Visual comparison among the competitors on the low-light image dataset.
Figure 3. Visual comparison among the competitors on the low-light image dataset.
Quantitative performance comparison on the low-light image dataset in terms of loe.
Table 1. Quantitative performance comparison on the low-light image dataset in terms of loe. loe has a factor 103. the lower the loe is, the better the enhancement preserves the naturalness of illumination.
Average running time comparison of the six enhancement methods on the low-light image dataset.
Table 2. Average running time comparison of the six enhancement methods on the low-light image dataset. The sizes of images are 1280 pixels (horizontal) × 960 pixels (vertical) and 4000 pixels (horizontal) × 3000 pixels (vertical).
Fang Lei presenting at WCCI2022
Fang Lei presenting at WCCI2022

Xuelong Sun presents research into the insect’s central complex in the midbrain for coordinating multimodal navigation behaviours at iNAV2022

The 4th Interdisciplinary Navigation Symposium, iNAV2022, was held fully virtual from the 14th to the 16th June  2022. This symposium mainly focused on the question: “How does the brain know where it is, where it is going, and how to get from one place to another?” Interestingly, this symposium took place in an 8bit 2D environment under the support of “Gather.town”, making it the most interactive virtual-academic-conference ever before in this special pandemic period.

ULRACEPT researcher Xuelong Sun presented a poster about his research on the insect’s central complex in the midbrain for coordinating multimodal navigation behaviours, ‘How the insect central complex could coordinate multimodal navigation’ Xuelong Sun; Shigang Yue; Michael Mangan. This poster appealed to several researchers sharing similar research interests who have further communicated with Xuelong for future directions of insect navigation. Xuelong has also answered some questions about the details of the neural networks applied in his work.

The Plenary talks were highly-qualified and overlapped with Xuelong’s research about insect navigation. Prof. Barbara Webb from the University of Edinburgh delivered a great talk about modelling the adaption in insect navigation which also mentioned Xuelong’s published paper. Xuelong said: “I am very happy that Webb mentioned my work. I have obtained many useful ideas and inspiration from the plenary talks and communications with peers, which will help my future research.”

ULTRACEPT Researchers Present at ICARM22

The IEEE International Conference on Advanced Robotics and Mechatronics (ICARM) is the flagship conference of both IEEE-SMC TC on Bio-mechatronics and Bio-robotics Systems, and IEEE-RAS TC on NeuroRobotics Systems. ICARM 2022 took place in the Steigenberger Hotel, Guilin, China from July 9th to 11th, 2022.  ULTRACEPT researchers Qinbing Fu, Xuelong Sun and Tian Liu attended this event with their co-authored paper titled: “Efficient bio-robotic estimation of visual dynamic complexity”.

ICARM22 presentation
ICARM22 presentation

Qinbing presented “Efficient bio-robotic estimation of visual dynamic complexity” in the regular session of the conference. This presentation gave a great introduction and demonstration of our multimodal swarm robotics platform named VColCOSP, which appealed to our academic peers who share similar researcher interests.

ICARM22 Qinbing Fu presenting
ULTRACEPT researcher Qinbing Fu presenting at ICARM22

Abstract

Visual dynamic complexity is ubiquitous, hidden attribute of the visual world that every motion-sensitive vision system is faced with. However, it is implicit and intractable which has never been quantitatively described due to difficulty in defending temporal features correlated to spatial image complexity. Learning from biological visual processing, we propose a novel bio-robotic approach to estimate visual dynamic complexity, effectively and efficiently, which can be used as a new metric for assessing dynamic vision systems implemented in robots. Here we apply a bio-inspired neural network model to quantitatively estimate such complexity associated with spatial-temporal frequency of moving visual scene. The model is implemented in an autonomous micro-mobile robot navigating freely in an arena encompassed by visual walls displaying moving scenes. The response of the embedded visual module can make reasonable prediction on surrounding dynamic complexity since it can be mapped monotonically to varying moving frequencies of visual scene. The experiments demonstrate this “predictor” is effective against different visual scenarios that can be established as a new metric for assessing visual systems. To prove its viability, we utilise it to investigate the performance boundary of a collision detection visual system in changing environment with increasing dynamic complexity.

The conference provided the ULTRACEPT researchers an opportunity to network and participate in knowledge exchange with researchers from other universities and academic institutions, including leading experts and professors in this field, paving the way for potential research cooperation in the future.

In this event, the ULTRACEPT group listened to high quality plenary talks relevant with our group’s topics. Xuelong said “the ideas presented by the speakers cover many aspects of the cutting-edge technologies of AI and autonomous robots such as swarm intelligence, embodiment, cognitive, etc.”. The group had an impressive experience and grasped some interesting and useful ideas and inspirations for future studies.

ICARM22 Xuelong Sun, Qinbing Fu, Tian Liu
ICARM22 attended by UTLRACEPT researchers Xuelong Sun, Qingbing Fu, Tian Liu

Machine Learning: Methods and Applications

To aid and support the continued collaboration and knowledge exchange of the ULTRACEPT researchers, the consortium hosts online quarterly ‘Sandpit Sessions’. The aim of these sessions is to provide researchers an opportunity to share their work in an informal forum where they can raise and discuss issues and challenges in order to gain support and feedback from the group.

The project group’s fifth sandpit session was hosted online by ULTRACEPT partner Westfälische Wilhelms-Universität Muenster (WWU) on the 17th of May 2022. The session also hosted a guest speaker from Chaoyang University of Technology (CYUT) in Taiwan. The session was chaired by Prof. Xiaoyi Jiang (WWU) and featured three speakers: Andreas Nienkötter, Postdoc, WWU; Jiaqi Zhang, PhD student, WWU; and Vani Saravanarajan, PhD student, CYUT. The theme of the session was ‘Machine Learning: Methods and Applications’.

Sandpit Session 5: Machine Learning: Methods and Applications

  • Date: 17th May 2022
  • Time: UK 11:00; China 18:00; Germany 12:00; Argentina 07:00; Malaysia 18:00; Japan 19:00.
  • Facilitators:
    • Andreas Nienkötter, Postdoc, WWU, Germany
    • Jiaqi Zhang, PhD student, WWU, Germany
    • Vani Saravanarajan, PhD student, CYUT, Taiwan
    • Xiaoyi Jiang, WWU, Germany (chair)
  • Location: MS Teams
Sandpit Schedule
UK Time Item Presenter/s
11:00-11:05 Arrival and welcome Xiaoyi Jiang
11:05-11:35 Generalized median computation for consensus learning

This talk presents the generalized median computation that is a particular form of consensus learning based on a mathematical optimization framework. Computational algorithms and potential applications will be briefly presented.

Andreas Nienkötter
11:35-12:00 Soft min/max and median estimation

This talk presents different ways of soft estimation of minimum, maximum, and median operator. Such soft estimation makes the computation differentiable and is thus helpful in many applications.

Jiaqi Zhang
12:00-12:25 Car crash detection using ensemble and transfer learning neural networks

This talk presents different types of neural networks and the ways to combine them for multi-class car crash detection.  The neural networks are trained with a small dataset and can be extended to many useful applications.

Vani Saravanarajan
12:25-12:30 Final comments Xiaoyi Jiang
Next session

Our next sandpit session will be held in September 2022 and hosted by Jiannan Zhao. More details will be provided closer to the time.

Andreas Nienkötter presented his talk on ‘Generalized median computation for consensus learning’.

Sandpit Session 5: Machine Learning: Methods and Applications
Andreas Nienkötter presents Generalized median computation for consensus learning

Jiaqi Zhang presented her talk on ‘Soft min/max and median estimation’.

Sandpit Session 5: Machine Learning: Methods and Applications
Jiaqi Zhang presents Soft min/max and median estimation

Vani Saravanarajan presented her talk on ‘Car crash detection using ensemble and transfer learning neural networks’.

Sandpit Session 5: Machine Learning: Methods and Applications
Vani Saravanarajan presents Car crash detection using ensemble and transfer learning neural networks

Jiaqi Zhang completes 6 month ULTRACEPT EU H2020 secondment in China

Westfälische Wilhelms-Universität Muenster (WWU) PhD researcher Jiaqi Zhang, recently completed a 6 month secondment for ULTRACEPT at partner Institute of Automation, Chinese Academy of Sciences (CASIA) in China from October 2021 to May 2022. Jiaqi shares her secondment experience in this blog post.

During my 6 month secondment at the Institute of Automation Chinese Academy of Sciences (CASIA), I focused on my research on interpolation kernel machine, which is a fundamental machine learning method that can be used for classification and regression problems.

Interpolation kernel machine is a general class of kernel-based techniques for classification and regression with several favorable properties. In its standard formulation, it can only deal with positive definite kernels. In real-world applications, however, one often has good reasons or even has to work with indefinite kernels.

While many kernel based machine learning methods, in particular support vector machines, have seen extensions to indefinite kernels, there was, to our knowledge, no previous work on indefinite interpolation kernel machines yet. We address this open issue and propose strategies and concrete implementations to make interpolation kernel machines functional for indefinite kernels, which is our main contribution. We conducted experiments on graph datasets from various domains and UCI datasets show competitive performance, also in comparison with the indefinite SVM counterparts. Currently, interpolation kernel machines have not received due attention, in particular, compared to SVM. Our work thus will help to increase the awareness of this powerful learning model in the community as well.

Year-end summary meeting at CASIA
Year-end summary meeting at CASIA

I am very appreciative of the ULTRACEPT project, Prof. Liu (CASIA), Prof. Jiang (WWU), and the EU H2020 funding which provided me the opportunity to work with CASIA. The Pattern Analysis and Learning group (PAL) has skilled and kind researchers, as well as significant research contributions.

During my 6 month secondment, I improved my skills in finding and solving problems. I learned about the research environment in China, which helped me to choose my future career directions. It also enabled me to understand the difference in research strategies between Germany and China, which can help me to enrich my own work skills and make decisions for my future career.

You can read more about Jiaqi’s research in her publication which she presented at the ICPRAI 2022 – 3rd International Conference on Pattern Recognition and Artificial Intelligence held June 1- 3, 2022 in Paris, France. She presented during the Special Session: Graphs for Pattern Recognition: Representations, Theory and Applications.

Zhang, J., Liu, CL., Jiang, X. (2022). Interpolation Kernel Machine and Indefinite Kernel Methods for Graph Classification. In: El Yacoubi, M., Granger, E., Yuen, P.C., Pal, U., Vincent, N. (eds) Pattern Recognition and Artificial Intelligence. ICPRAI 2022. Lecture Notes in Computer Science, vol 13364. Springer, Cham. https://doi.org/10.1007/978-3-031-09282-4_39

Jiaqi Zhang on secondment at CASIA
Jiaqi Zhang on secondment at CASIA

Attention and Prediction-Guided Motion Detection for Low-Contrast Small Moving Targets

ULTRACEPT University of Lincoln researcher Hongxin Wang recently published a paper titled “Attention and Prediction-Guided Motion Detection for Low-Contrast Small Moving Targets” in IEEE Transactions on Cybernetics. IEEE Transactions on Cybernetics is one of the top-tier journals that publish technical articles dealing with communication and control across machines, humans, and organizations. It has a significant influence on machine learning, fuzzy systems, cognitive systems, decision making, and robotics, to the extent that they contribute to the theme of cybernetics or demonstrate an application of cybernetics principles.

H. Wang, J. Zhao, H. Wang, C. Hu, J. Peng and S. Yue, Attention and Prediction-Guided Motion Detection for Low-Contrast Small Moving Targets, in IEEE Transactions on Cybernetics,  https://doi.org/10.1109/TCYB.2022.3170699

Research Summary

Small target motion detection within complex natural environments is an extremely challenging task for autonomous robots. Surprisingly, the visual systems of insects have evolved to be highly efficient in detecting mates and tracking prey, even though targets occupy as small as a few degrees of their visual fields. The excellent sensitivity to small target motion relies on a class of specialized neurons, called small target motion detectors (STMDs). However, existing STMD-based models are heavily dependent on visual contrast and perform poorly in complex natural environments, where small targets generally exhibit extremely low contrast against neighboring backgrounds. In this research, we develop an attention-and-prediction-guided visual system to overcome this limitation. The developed visual system comprises three main subsystems, namely: 1) an attention module; 2) an STMD-based neural network; and 3) a prediction module, as shown in Fig. 1. The attention module searches for potential small targets in the predicted areas of the input image and enhances their contrast against a complex background. The STMD-based neural network receives the contrast-enhanced image and discriminates small moving targets from background false positives. The prediction module foresees future positions of the detected targets and generates a prediction map for the attention module. The three subsystems are connected in a recurrent architecture, allowing information to be processed sequentially to activate specific areas for small target detection.

Research Highlights

To overcome heavy dependence of the existing models on visual contrast between small targets and the background, this research develops an attention and prediction guided visual system. Prediction and attention are fundamental functions in the visual systems of insects, where the former utilizes present and/or past information to anticipate future object motion, while the latter prioritizes objects of interest amidst a swarm of potential alternatives. In the proposed visual system, an attention module and a prediction module are connected with an STMD-based neural network in a recurrent architecture. At each time step, the input image and a prediction map are applied to the attention module to search for potential small targets in several predicted areas. A contrast-enhanced image is produced by enhancing the contrast of potential targets over the input image, and then fed into the STMD-based neural network for discriminating small moving targets. The prediction module anticipates future positions of the detected small targets and generates a prediction map which is propagated to the attention module in the next time step. Extensive experiments on synthetic and real-world datasets demonstrate the effectiveness and superiority of the proposed visual system for detecting small, low contrast moving targets against complex natural environments, as shown in Fig. 2.

Overall flowchart of the proposed attention and prediction guided visual system
Fig. 1. Overall flowchart of the proposed attention and prediction guided visual system. It consists of a preprocessing module (left), an attention module (top), an STMD-based neural network (right), a prediction module (bottom), and a memorizer (middle).
Representative frames of the three synthetic videos that hold multiple small targets moving against complex backgrounds and corresponding ROC curves of the four models.
Fig. 2. (a)-(c) Representative frames of the three synthetic videos that hold multiple small targets moving against complex backgrounds and corresponding ROC curves of the four models. The proposed model consistently performs best among the four different methods for multiple small target detection against different background images.

 

A Multiple Pheromone Communication System for Swarm Intelligence

ULTRACEPT researcher Mr. Tian Liu recently published his journal article T. Liu, X. Sun, C. Hu, Q. Fu and S. Yue, A Multiple Pheromone Communication System for Swarm Intelligence, in IEEE Access, vol. 9, pp. 148721-148737, 2021. In this post, Mr. Liu shares with us the highlights of this research.

ColCOSPhi

Research Summary

Pheromones are chemical substances essential for communication among social insects. In the application of swarm intelligence to real micro mobile robots, the deployment of a single virtual pheromone has emerged recently as a powerful real-time method for indirect communication. However, these studies usually exploit only one kind of pheromones in their task, neglecting the crucial fact that in the world of real insects, multiple pheromones play important roles in shaping stigmergic behaviours such as foraging or nest building. To explore the multiple pheromones mechanism which enable robots to solve complex collective tasks efficiently, we introduce an artificial multiple pheromone system (ColCOSΦ) to support swarm intelligence research by enabling multiple robots to deploy and react to multiple pheromones simultaneously. The proposed system ColCOSΦ uses optical signals to emulate different evaporating chemical substances i.e. pheromones. These emulated pheromones are represented by trails displayed on a wide LCD display screen positioned horizontally, on which multiple miniature robots can move freely. The colour sensors beneath the robots can detect and identify lingering “pheromones” on the screen. Meanwhile, the release of any pheromone from each robot is enabled by monitoring its positional information over time with an overhead camera. No other communication methods apart from virtual pheromones are employed in this system. Two case studies have been carried out which have verified the feasibility and effectiveness of the proposed system in achieving complex swarm tasks as empowered by multiple pheromones. This novel platform is a timely and powerful tool for research into swarm intelligence.

Research Highlights

This research introduced novel platform which can implement swarm robots experiments with multiple pheromone communication. One of the key component to utilize this system is the micro-robot platform which can detect, identify and react to the optically emulated pheromones on the screen. We have designed an ideal micro-robot platform based on the Colias-IV robot prototype which is a differential driven ground robot featured with small in size, light-weight and strong computing power.

In this study, a series of experiments have been designed using real robots to demonstrate the feasibility of the proposed system, followed by two case studies including a food recruitment task and a behaviour mediation task. These bespoke case studies investigate the efficiency, modulation method and robustness of the system. Results from these systematic experiments show that multiple pheromones can be precisely emulated by optical signals. Their functionalities can also be interpreted and reacted correctly by swarm robots in real-time. The proposed multiple pheromone system provides a powerful tool facilitating the exploration of more complex collective behaviours in both biological and robotic swarm systems, especially in foraging and aggregation.

Future research of multiple pheromone mechanism may play an important role in unravelling the mechanism underling complex collective behaviours and could potentially provide bio-inspired solutions for challenging problems in swarm robotics.