Current Research Projects

with Gergely Neu,
Tomáš Kocák,
Rémi Munos,
Branislav Kveton,
Shipra Agrawal:
Graph and Combinatorial Bandits, 2013  present
We are interested in efficient sequential algorithms for complex action spaces. For example, the set of actions can be graph (users in a social networks, movies in the recommender systems) or a large combinatorial space (set of all paths from A to B in a given graph). When appropriate, we seek to use similarity properties  closeby actions exhibit similar behavior. Moreover, we also make use of side observations (information from the connections in social networks). [ICML 2014] [AAAI 2014] [NIPS 2014] [NIPS 2014] [ICML 2015]

with JeanBastien Grill,
Rémi Munos,
Nathaniel Korda,
Alexandra Carpentier:
Structured Bandit problems, 2011  2015,
Bandit Learning
Leveraging structure properties in the bandits problems with large number of arms. For example, the rewards of arms may be smooth function of their contexts as given by some similarity function (kernel). In the more challenging case, the reward function may be only locally smooth around one of the best arms and this smoothness is unknown. [UAI 2013] [ICML 2013] [ICML 2015] [NIPS 2015]

with Julien Audiffren, Mohammad Ghavamzadeh, and Alessandro Lazaric:
Semisupervised apprenticeship learning, 2011  2015, relevant literature
In apprenticeship learning we aim to learn a good behavior by observing an expert or a set of experts. We assume a setting where the expert is maximizing an unknown true reward function, which is often a linear combination of known state features. We consider a situation when we observe many trajectories of behaviors but only one or a few of them are labeled as experts' trajectories. We investigate the assumptions under which the remaining unlabeled trajectories can aid in learning a policy with a good performance. [EWRL 2012] [IJCAI 2015]  with Daniele Calandriello,
Alessandro Lazaric,
Branislav Kveton (Technicolor), Avneesh Saluja (CMU):
Largescale semisupervised learning 2010  present
We parallelized online harmonic solver to process 1 TB of video data in a day. I am working on the multimanifold learning that can overcome changes in distribution. I am showing how the online learner adapts as to characters' aging over 10 years period in Married ... with Children sitcom. The research was part of Everyday Sensing and Perception (ESP) project. [FG 2013]
Recent Research Projects

with Rémi Munos,
Mohammad Ghavamzadeh,
Alessandro Lazaric, and
Daniil Ryabko:
Composing Learning for Artificial Cognitive Systems 2011  2015
The purpose of this project is to develop a unified toolkit for intelligent control in many different problem areas. This toolkit will incorporate many of the most successful approaches to a variety of important control problems within a single framework, including bandit problems, Markov Decision Processes (MDPs), Partially Observable MDPs (POMDPs), continuous stochastic control, and multiagent systems. [EWRL 2012] [UAI 2013] [ICML 2013] [ICML 2014] [AAAI 2014] [NIPS 2014] [NIPS 2014] [NIPS 2014]  with Branislav Kveton:
Online semisupervised learning 2009  2010
I extended graphbased semisupervised learning to the structured case and demonstrated on handwriting recognition and object detection from video streams. Regularized harmonic function solution: The algorithm outputs a confidence of inference and uses it for learning. I came up with an online algorithm that on the realworld datasets recognizes faces at 8090% precision with 90% recall. [UAI 2010] [AISTATS 2010] [CVPR  OLCV 2010]  with Miloš Hauskrecht:
Anomaly detection in clinical databases 2007  2013
Statistical anomaly detection methods for identification of unusual outcomes and patient management decisions. I combined maxmargin learning with distance learned to create and anomaly detector, which outperforms the hospital rule for Heparin Induced Thrombocytopenia detection. I later scaled the system for 5K patients with 9K features and 743 clinical decisions per day. At the recent study, from 222 alerts 50% were highly relevant. [AMIA 2007] [FLAIRS 2008] [ICMLHEALTH 2008] [MEDINFO 2010] [AMIA 2010] [ICDM 2011] [JBI 2013]  with Wendy Chapman,
Roger Day and
Gregory Cooper:
OddManOut 2007  2011
We hypothesized that clinical data in emergency department (ED) reports would increase sensitivity and specificity of case identification for patients with an acute lower respiratory syndrome (ALRS). We designed a statistic of disagreement (oddmanout) to evalute the machine learning classifier with expert evaluation in the cases when the gold standard is not available. [ISDS 2006]  with Miloš Hauskrecht, Richard Pelikan, and Shuguang Wang
Highthroughput proteomic and genomic data and biomarker discovery 2006  2007
I built a framework for the cancer prediction from highthroughput proteomic and genomic data sources. I found a way to merge heterogeneous data sources: My fusion model was able to predict pancreatic cancer from Luminex combined with SELDI with 91.2% accuracy. [Springer 2006] [STB 2008] [IJD 2011]
Past Research Projects
 with Nuno Miguel Cavalheiro Marques and
Marco Castelani:
Evolutionary feature selection algorithms 2005
I enhanced the existing FeaSANNT neural feature selection with spiking neuron model to handle inputs noised with up to 10% Gaussian noise.
 with Juraj Pavlásek Radoslav Harman and Ján Jenča: Plastic Synapses (regularity counting) 2003  2005
I was modelling basic learning function at the level of synapses. I designed a model that is able to adapt to the regular frequencies with different a rate as the time flows. I used genetic programming to find biologically plausible networks that distinguish different gamma distribution and provided explanation of the strategies evolved.
8Nov2015