Linguistic spatial reasoning
Speaker(s) : James M. Keller (University of Missouri, USA and LIP6)
With the collaboration of several faculty colleagues and many students, I have been studying the creation and utilization of spatial relations in various sensor-related domains for many years. Scene description, involving linguistic expressions of the spatial relationships between image objects, is a major goal of high-level computer vision. In a series of papers, we introduced the use of histograms of forces to produce evidence for the description of relative position of objects in a digital image. There is a parameterized family of such histograms, for example, the histogram of constant forces (much like the earlier histogram of angles) and the histogram of gravitational forces that highlights areas that are close between the two objects. Utilizing the fuzzy directional membership information extracted from these histograms within fuzzy logic rule-based systems, we have produced high-level linguistic descriptions of natural scenes as viewed by an external observer. Additionally, we have exploited the theoretical properties of the histograms to match images that may be the same scene viewed under different pose conditions. In fact, we can even recover estimates of the pose parameters. These linguistic descriptions have then been brought into an ego-centered viewpoint for application to robotics, i.e., the production of linguistic scene description from a mobile robot standpoint, spatial language for human/robot communication and navigation, and understanding of a sketched route map for communicating navigation routes to robots. This last activity can be labeled as Sketch-to-Text.
Recently, with a grant from the National Geospatial Intelligence Agency and collaboration with personnel at the Institute for Human and Machine Cognition, we are tackling the inverse problem: given one or more text descriptions of a temporal and spatial event, construct a sketch of the event for subsequent reasoning. This is called Text-to-Sketch by the NGA. The idea is that the person or persons providing the linguistic descriptions either may not know where they are exactly or they may not use referenced landmarks in the descriptions. Hence, the input may only be a temporal sequence of objects and their relationships. The produced graphics-based sketch must be grounded in reality by matching it to a satellite image or geospatial database. The techniques involve natural language understanding, fuzzy approaches and force histogram matching for intelligent sketch production, and subgraph isomorphism algorithms and/or genetic algorithms for sketch to geospatial database matching. This talk will quickly survey the early applications and focus on our approach to the new problem.
James M. Keller received the Ph.D. in Mathematics in 1978. He holds the University of Missouri Curators’ Professorship in the Electrical and Computer Engineering and Computer Science Departments on the Columbia campus. He is also the R. L. Tatum Professor in the College of Engineering. His research interests center on computational intelligence: fuzzy set theory and fuzzy logic, neural networks, and evolutionary computation with a focus on problems in computer vision, pattern recognition, and information fusion including bioinformatics, spatial reasoning in robotics, geospatial intelligence, sensor and information analysis in technology for eldercare, and landmine detection. His industrial and government funding sources include the Electronics and Space Corporation, Union Electric, Geo-Centers, National Science Foundation, the Administration on Aging, The National Institutes of Health, NASA/JSC, the Air Force Office of Scientific Research, the Army Research Office, the Office of Naval Research, the National Geospatial Intelligence Agency, the Leonard Wood Institute, and the Army Night Vision and Electronic Sensors Directorate. Professor Keller has coauthored over 350 technical publications.
Jim is a Fellow of the Institute of Electrical and Electronics Engineers (IEEE) for whom he has presented live and video tutorials on fuzzy logic in computer vision, is an International Fuzzy Systems Association (IFSA) Fellow, an IEEE Computational Intelligence Society Distinguished Lecturer, a national lecturer for the Association for Computing Machinery (ACM) from 1993 to 2007, and a past President of the North American Fuzzy Information Processing Society (NAFIPS). He received the 2007 Fuzzy Systems Pioneer Award from the IEEE Computational Intelligence Society. He finished a full six year term as Editor-in-Chief of the IEEE Transactions on Fuzzy Systems, is an Associate Editor of the International Journal of Approximate Reasoning, and is on the editorial board of Pattern Analysis and Applications, Fuzzy Sets and Systems, International Journal of Fuzzy Systems, and the Journal of Intelligent and Fuzzy Systems. Jim was the Vice President for Publications of the IEEE Computational Intelligence Society from 2005-2008, and is currently an elected Adcom member. He was the conference chair of the 1991 NAFIPS Workshop, program co-chair of the 1996 NAFIPS meeting, program co-chair of the 1997 IEEE International Conference on Neural Networks, and the program chair of the 1998 IEEE International Conference on Fuzzy Systems. He was the general chair for the 2003 IEEE International Conference on Fuzzy Systems.
Thomas.Baerecke (at) nulllip6.fr