
%%%%%%%%%%%%%%%%%%%%%%% file typeinst.tex %%%%%%%%%%%%%%%%%%%%%%%%%
%
% This is the LaTeX source for the instructions to authors using
% the LaTeX document class 'llncs.cls' for contributions to
% the Lecture Notes in Computer Sciences series.
% http://www.springer.com/lncs       Springer Heidelberg 2006/05/04
%
% It may be used as a template for your own input - copy it
% to a new file with a new name and use it as the basis
% for your article.
%
% NB: the document class 'llncs' has its own and detailed documentation, see
% ftp://ftp.springer.de/data/pubftp/pub/tex/latex/llncs/latex2e/llncsdoc.pdf
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


\documentclass[runningheads,a4paper]{llncs}

\usepackage{amssymb}
\setcounter{tocdepth}{3}
\usepackage{graphicx}
\usepackage{bm}

\usepackage{url}
\urldef{\mailsa}\path|{alfred.hofmann, ursula.barth, ingrid.haas, frank.holzwarth,|
\urldef{\mailsb}\path|anna.kramer, leonie.kunz, christine.reiss, nicole.sator,|
\urldef{\mailsc}\path|erika.siebert-cole, peter.strasser, lncs}@springer.com|
\newcommand{\keywords}[1]{\par\addvspace\baselineskip
\noindent\keywordname\enspace\ignorespaces#1}

\begin{document}

\mainmatter  % start of an individual contribution

% first the title is needed
\title{CS224W Project Milestone - \\ Understanding Human Actions: An Information Network Perspective of View}

% a short form should be given in case it is too long for the running head
\titlerunning{CS224W Project Milestone}

% the name(s) of the author(s) follow(s) next
%
% NB: Chinese authors should write their first names(s) in front of
% their surnames. This ensures that the names appear correctly in
% the running heads and the author index.
%
\author{Bangpeng Yao, Bo Wang, Cherhan Foo}
%
\authorrunning{CS224W Project Milestone}
% (feature abused for this document to repeat the title also on left hand pages)

% the affiliations are given next; don't give your e-mail address
% unless you accept that it will be published
\institute{Stanford University}

%
% NB: a more complex sample for affiliations and the mapping to the
% corresponding authors can be found in the file "llncs.dem"
% (search for the string "\mainmatter" where a contribution starts).
% "llncs.dem" accompanies the document class "llncs.cls".
%

\toctitle{Lecture Notes in Computer Science}
\tocauthor{Authors' Instructions}
\maketitle

%\begin{abstract}
%abstract.
%\end{abstract}


\section{Problem Formulation and Related Work}

Recognizing human actions is a hot topic in both psychology~\cite{Gentner82} and computer vision~\cite{Yao10}. Most of the works treat action recognition as a pure classification problem, especially in computer vision. Typically, people have a list of pre-defined actions (usually no more than 15), and then collect a set of images and annotate the humans that are doing one of the pre-defined actions. Then the rest of the story is all about classification.

However, there are a number of actions in real life. According to a survey done by the American Bureau of Labor~\cite{Activity10}, there are at least hundreds of actions that happens frequently in human life, and those actions can be further divided into some sub-categories. For example, ``changing sheets'' and ``sweeping floors'' are ``housework'' actions, while ``riding a bike'' and ``high jumping'' are sports actions.

The goal of our project is to study the relationships between different human actions, and give explanations to the relationships. As mentioned in~\cite{Gentner82}, human actions refer to the concepts that are abstract, unlike objects that are well defined. To be more specific, human actions are related to the spatial configurations of human body parts (head, arms, legs, etc.) and their interactions between the environment, where the human poses can change continuously. Humans discretize this space using some words or phrases of verbs and nouns, without knowing why some human poses and the interactions between humans and objects are described by the same verb or phrase of verbs and nouns. The human poses in different actions can be very similar, whereas in some situations the poses in the same action can be significantly different, as shown in Fig.\ref{fig:pose_demo}.

\begin{figure}[t]
\centering
\includegraphics[width=0.99\linewidth]{./figure/pose_demo.eps}
\caption{The left two images have similar human poses but they belong to different actions, while the right two images have very different human poses but belong to the same action.}\label{fig:pose_demo}
\end{figure}

In this project, we would like to study the properties of different human actions. Instead of treating action recognition as a pure image classification approaches, we would like to analyze the distributions of human poses and object that interaction with the human for each action, as well as the relationship between different actions. For instance, we would like to automatic uncover the sets of actions from the images with an unsupervised approach. Also the actions that interact with the same or related objects, or with similar human poses are considered related.

However, this is a challenging problem. As shown in Fig.\ref{fig:bike}, even when humans interact with the same object, the distributions of human poses can be significantly different. In this project, we will achieve our goal using information network analysis techniques.

\begin{figure}[t]
\centering
\includegraphics[width=0.99\linewidth]{./figure/bike.eps}
\caption{Even in the actions where humans interact with the same object, the distributions of human poses can be largely different. For instance, the human poses in images of ``riding a bike'' are usually similar, whereas the poses in ``repairing a bike'' are very different.}\label{fig:bike}
\end{figure}

\section{Key Algorithms}

We consider human actions as human poses (specific layouts of human body parts) and the objects that interact with the human (e.g. ``horse'' in ``riding a horse'', ``dish'' in ``washing dishes''). We further decompose an action image into a set of {\em poselets}~\cite{Bourdev09} and {\em objects}. Each poselet is a set of local human poses that are tightly consistent in terms of layout of human body parts, as shown in Fig.\ref{fig:poselet}. Based on this decomposition, we represent action images as a network, where each node represents a poselet or an object in a specific image location, as shown in Fig..

\begin{figure}[t]
\centering
\includegraphics[width=0.9\linewidth]{./figure/poselet.eps}
\caption{Examples of poselets. Images in each row are encoded by the same poselet.}\label{fig:poselet}
\end{figure}


\section{Progress}

\section{Next Steps}


\begin{thebibliography}{4}

\bibitem{Activity10} American Time Use Survey (ATUS) Activity Lexicon. Bureau of Labor Statistics, 2010.
    
\bibitem{Bourdev09} Bourdev, L., Malik, J.: Poselets: Body Part Detectors Trained Using 3D Human Pose Annotations. International Conference on Computer Vision (ICCV), 2009.

\bibitem{Gentner82} Gentner, D.: Why Nouns Are Learned Before Verbs: Linguistic Relativity Versus Natural Partitioning. Language Development, Volume 2: Language Thought and Cluture. Hillsdale, NJ, 1982.
    
\bibitem{Yao10} Yao, B., Fei-Fei, L.: Grouplet: A Structural Image Representation for Recognizing Human and Object Interactions. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010.

\end{thebibliography}

\end{document}
