Reinforcement Learning

Learning in reality is rarely completely autonomous and on isolated problems. I am interested in several distinct (but complementary) ways of helping reinforcement learning agents to arrive at solutions quicker, by augmenting their problems with extra knowledge (reward shaping), or their solutions with other agents’ solutions (ensemble learning).

Selected publications:

Expressing Arbitrary Reward Functions as Potential-Based Advice
Anna Harutyunyan, Sam Devlin, Peter Vrancx, and Ann Nowé
In Proceedings of the Twenty-Ninth Conference on Artificial Intelligence (AAAI), pp. 2652-2658, 2015.
»   [Paper]     [Slides]     [Bibtex]

@inproceedings{harutyunyan2015aaai,
title={{Expressing Arbitrary Reward Functions as Potential-Based Advice}},
author={Anna Harutyunyan and Sam Devlin and Peter Vrancx and Ann Now\'{e}},
booktitle={{Proceedings of the Twenty-Ninth Conference on Artificial Intelligence (AAAI)}},
pages ={2652-2658},
year={2015},
area={Reinforcement Learning},
type = {Refereed Conference},
acc_rate = {26.67\%},
acc_numbers = {531\textbackslash 1991},
}

Multi-Scale Reward Shaping via an Off-Policy Ensemble
Anna Harutyunyan, Tim Brys, Peter Vrancx, and Ann Nowé
In Proceedings of the Fourteenth International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), pp. 1641-1642, 2015.
»   [Paper]     [Bibtex]

@InProceedings{harutyunyan2015aamas,
author = {Anna Harutyunyan and Tim Brys and Peter Vrancx and Ann Now\'{e}},
title = {{Multi-Scale Reward Shaping via an Off-Policy Ensemble}},
booktitle = {{Proceedings of the Fourteenth International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS)}},
pages = { 1641-1642 },
year = {2015},
area={Reinforcement Learning},
type = {Refereed Conference (Short)},
acc_rate = {46.8\%},
acc_numbers = {167 of 670 + additional 147 short papers},
}

Assistive Robotics

I am part of the cross-disciplinary SBO project MIRAD: an integrated Methodology to bring Intelligent Robotic Assistive Devices to the user, the goal of which is to build bilateral intelligent active lower-limb exoskeleton to assist persons suffering from functional weakness.

My main role is in performing prediction to varying degrees: from the high-level intention of the user, to specific micro-events during different activities. For this, I’ve been mainly working with neural nets and Hidden Markov Models. We have also been interested in probabilistic modeling of the motion trajectories, using Gaussian Processes.

Publications:

Predicting Seat-Off and Detecting Start-of-Assistance Events for Assisting Sit-to-Stand with an Exoskeleton
K. Tanghe, A. Harutyunyan, E. Aertbelien, F. De Groote, J. De Schutter, P. Vrancx, and A. Nowé
In Robotics and Automation Letters, IEEE, pp. 792-799, 2016.
»   [DOI]     [Bibtex]

@ARTICLE{tanghe2016ieee,
author={Tanghe, K. and Harutyunyan, A. and Aertbelien, E. and De Groote, F. and De Schutter, J. and Vrancx, P. and Now\'{e}, A.},
journal={{Robotics and Automation Letters, IEEE}},
title={{Predicting Seat-Off and Detecting Start-of-Assistance Events for Assisting Sit-to-Stand with an Exoskeleton}},
year={2016},
volume={1},
number={2},
pages={792 - 799},
keywords={Physical human robot interaction;exoskeleton;physically assistive devices},
doi={10.1109/LRA.2016.2530165},
ISSN={2377-3766},
area={Exoskeleton Applications},
type={Journal}
}

Planar Graphs

During my Masters, I was working on graph theory research, specifically on problems of finding maximum flow in planar graphs. My focus was broadly to design algorithms that are practical, and whose analytic running times are meaningful.

Worst-case analysis is often uninformative in practice, as some problems may never reach the extreme worst-case complexity, while others have large hidden constants, and will fare worse in practice than their asymptotic running times suggest. We are thus interested in adaptive analysis techniques, that are expressive in terms of the problem instance. With this motivation, we design practical max flow algorithms that only make use of  simple data structures, and are easy to implement.

Related publications:

Boundary-to-boundary flows in planar graphs
Glencora Borradaile and Anna Harutyunyan
In Proceedings of the International Workshop on Combinatorial Algorithms (IWOCA), pp. 67-80, 2013.
»   [Paper]     [DOI]     [Bibtex]

@inproceedings{bh2013b-iwoca,
author = {Borradaile, Glencora and Harutyunyan, Anna},
booktitle = {{Proceedings of the International Workshop on Combinatorial Algorithms (IWOCA)}},
url = {http://dx.doi.org/10.1007/978-3-642-45278-9_7},
doi = {10.1007/978-3-642-45278-9_7},
isbn = {978-3-642-45277-2},
pages = {67-80},
publisher = {Springer},
series = {Lecture Notes in Computer Science},
title = {Boundary-to-Boundary Flows in Planar Graphs},
volume = 8288,
year = 2013,
area={Planar Graphs},
type = {Refereed Workshop and Symposia},
acc_rate = {36.4\%},
acc_numbers = {32 of 88}
}

Maximum st-Flow in Directed Planar Graphs via Shortest Paths.
Glencora Borradaile and Anna Harutyunyan
In Proceedings of the International Workshop on Combinatorial Algorithms (IWOCA), pp. 423-427, 2013.
»   [Paper]     [DOI]     [Bibtex]

@inproceedings{bh2013a-iwoca,
author = {Borradaile, Glencora and Harutyunyan, Anna},
booktitle = {{Proceedings of the International Workshop on Combinatorial Algorithms (IWOCA)}},
url = {http://dx.doi.org/10.1007/978-3-642-45278-9_36},
doi = {10.1007/978-3-642-45278-9_36},
isbn = {978-3-642-45277-2},
pages = {423-427},
publisher = {Springer},
series = {Lecture Notes in Computer Science},
title = {{Maximum st-Flow in Directed Planar Graphs via Shortest Paths.}},
volume = 8288,
year = 2013,
area={Planar Graphs},
type = {Refereed Workshop and Symposia},
acc_rate = {36.4\%},
acc_numbers = {32 of 88}
}

Masters thesis:

Maximum flow in planar digraphs
Anna Harutyunyan
.
»   [Bibtex]

@book{harutyunyan2012masters,
author = {Anna Harutyunyan},
title = {Maximum Flow in Planar Digraphs},
school={Oregon State University},
year={2012},
month={November},
address={Corvallis, OR, USA},
type={thesis}
}