Università degli Studi di Genova Dipartimento di Ingegneria Biofisica ed Elettronica Sistemi Cognitivi per le Telecomunicazioni Prof. Carlo Regazzoni DIBE
Why Cognitive Systems? In the last years the telecommunications market evolved from simple services (voice/slow rate data) towards very complex and dynamics services. These kinds of services require a strong interaction with human users. Users should be able to access services and information everywhere. 2
Why Cognitive Systems? Keywords for the next generation telecommunications/information provisioning services are: Mobility everywhere access Transparency users can simply access the services without any effort on system configurations Personalization the user can have services according to his/her preferences. Unobtrusivity the services should be provided in a natural and they should not annoy the user or distract the user in his/her common activities Pervasivity services should cover various aspects of everyday life of the user
Why Cognitive Systems? Cognitive systems can easily cover all the requirements of modern telecommunication systems. They can provide natural interactions with human users. They can offer a complete transparency thanks to their self-adaptability. They can be implemented at every level of the communication system architecture. They can be a useful and intelligent support to everyday life.
What is a Cognitive System? Let us give a general definition for Cognitive Systems: They include both natural and artifical information processing systems responsible for perception, learning, reasoning, communication, action, and more. [1] Humans and animals, are clear examples of natural cognitive systems. 5
Cognitive Cycle The common shared model for describing the behavior of a Cognitive System is the so-called Cognitive Cycle and it is composed by 4 main characteristics. DECISION ACTION E ANALYSIS SENSING 6
Cognitive Cycle Sensing is the passive interaction component: the system has to continuously acquire knowledge about the interacting objects and about its own internal status DECISION ACTION E ANALYSIS SENSING 7
Cognitive Cycle Analysis perceived raw data need an analysis phase to represent them and extract interesting filtered information DECISION ACTION E ANALYSIS SENSING 8
Cognitive Cycle Decision the intelligence of the system is expressed by the ability to decide for the proper action, given a basic knowledge, experience and sensed data DECISION ACTION E ANALYSIS SENSING 9
Cognitive Cycle Action expresses the active interaction of the System can take in relation to its decision. The system tries to influence its interacting entities to maximize the functional of its objective DECISION ACTION E ANALYSIS SENSING 10
Learning A Cognitive System is characterized by the ability of improving its behavior through the experiences directly lived or only observed. The Learning phase is continuous and involves all the stages (within certain limits) of the Cognitive Cycle. As a matter of fact, the physical capabilities of the body limits the potential actions the system can do. The Intelligence of the system (the analysis and decision stages or Cognitive Core) are potentially unlimited in their evolution and improvement 11
Knowledge Representation It is possible to divide the artificial cognitive systems into two main macro-classes: Logical Artificial Cognitive Systems Bio-Inspired Artificial Cognitive Systems Each class is characterized by a different way of representing and processing the information within the cognitive cycle. 12
Logical Cognitive Systems Logical Cognitive Systems follow the typical architectural structure of the GOFAI systems (Good ( Intelligence Old Fashoned Artificial This research trend was born during the Second World War with the works of Alan Turing and reached the maximum evolution during the 80s with Allen Newell [2]. The knowledge is represented at a semantic level, i.e. it is formed by labels (words) that can be easily understood by human operators 13
Logical Cognitive Systems All the knowledge has to be available to the cognitive systems when it is swhitched on. This means that all the possible cases and situations the system could face have to be foreseen and to each situation there should be associated a precodified action (Expert Systems [3]). DRAWBACKS: This kind of structure strongly limits the potentiality of learning of the system. It is impossible to foresee all the situations: this fact could lead to system deadlocks. 14
Example of Logical Cognitive System if Mary is in the room { if Mary is moving { Wait for Mary s inputs } else { if Mary is lying { Mary is sleeping then { if the light is switched on { turn the light off } else IDLE } else if Mary is sitted and Mary has a book { Mary is studying then { if the phone is switched on { turn the phone off } else IDLE } } else IDLE 15
Example of Logical Cognitive System In blue there are the statement logical operators that link one another the various states the components of the system can assume. Each state can be TRUE or FALSE according to the binay logic. New visions of the AI use a feature space instead of the semantic state labels. As a matter of fact, features are more related to the physicality of the specific problem the AI system have to solve This last method allowed to provide more flexibility to the state description but the main AI drawbacks still remain. 16
Bio-Inspired Cognitive Systems Bio-Inspired Cognitive Systems try to build the intelligence of the system starting from the physiological basic functioning of a living entity. This research trend started from the studies of McCullok [4] on the animal nervous system. His theories were fundamental in the definition of the so-called Artificial Neural Networks (introduced by Rosenblatt [5]). The comprehension of the interaction and coordination among huge sets of neurons brought to the definition of new interesting paradigms for building up Artificial Cognitive Systems. 17
Bio-Inspired Cognitive Systems Robotics AI scientist Rodney Brooks firstly addressed the paradigm of Embodied Cognition-based Systems [6]. The main assumption is that the system is aware of the physical possibilities and limits of its body. This theory founds a confirmation in the evolution of the human intelligence. R. Llinas hypothesizes that the motor possibilities of the ancestral body drove the evolution of the human brain [7]. 18
Bio-Inspired Cognitive Systems The construction of the algorithms that implement the cognitive cycle is based on the specific application/body of the system. Different Sensing capabilities lead to different Analysis methodologies. Different Acting capabilities lead to different Decisions that can be taken. Intelligence DECISION ACTION Motor Muscles Physicality BRAIN BODY ANALYSIS SENSING Perceptive Organs
The Artificial Cognitive System Once defined the different models of intelligence in cognitive entities, let us focus the attention on how designing an Artificial Cognitive System. Let us define the characteristics of the artificial body. Let us define which issues the artificial mind have to face in order to perform complete cognitive tasks.
How the artificial body is? Human/Machine Interface ( Video/Audio ) Memory Audio Sensors Physical Actuators Radio/E.M. Sensors Physical Sensors Radio Actuators Video Sensors Other Software Actuators Processing Unit Other Software Sensors
Issues for the Artificial Mind Starting from the capabilities of the body, the artificial mind should be composed by algorithms and methods for: Acquiring and pre-process physical signals. Acquiring and pre-process other perceptual information. Extracting synthetic characteristics from the acquired information/signals. Classifying in which context the cognitive systems is. Fusing Information coming from multiple sensors/other cooperating cognitive entities. Deciding which is the proper action to pursue. Transducing the decide action into physical/software simple drivers/triggers.
Cognitive Cycle Analysis perceived raw data need an analysis phase to represent them and extract interesting filtered information DECISION ACTION E ANALYSIS SENSING 23
Applications for Cognitive systems Context-based content/service tagging Multimedia album, Automatic blog, E-tourism Context-based content/service adaptation Enriched personal communication Context-based content/service advertising/recommendation E-tourism, Content/Service Push Context Advertising Context sharing Enriched presence 24
Tourism and Transport Transports: User needs updated information about surrounding environment traffic jams public transport network Tourism: Information about places of interest by location user preferences 25
TecHotel Services: Automatic check-in City events Service bookings (..., sitter,restaurant (personal trainer, baby Neighbour services (..., stations,airport (rail and metro Available connectivity External content provider access (... Michelin (Guida 26
Mass Market Distribution/Events Crowd of users Time optimization ( selection (best route Advertising of services and products Real-time stream of information Queue manager 27
Infotainment Information News, sports, politics, traffic forecast, weather forecast, advertising, info on specific user interests Entertainment Games, jokes, music and pictures download, jokes etc... 28
Work Force Management Specific features for working-areas support and management Command, control and information Data and operative instructions management, team making and handling, assignment to specific tasks Training on-the-job Interactive training, on-demand data support, real-time connections between workers 29
Context Tagging 30
Context Tagging Find useful informations about visited places (... guides Tourism (monuments, churches, on-line (... stations Services (restaurants, metro, bus Localization (on map, find other users nearby...) Context sharing Upload photos and videos on the internet in real-time Create scrap books accessible to you and your friends Define a profile of your interests 31
Information Processing Hierarchical decomposition in 3 different layers Services decomposed in subtasks Different levels of abstraction Data fusion 32
Information processing: subtasks Input: multisensory data stream Sub-task functionalities: Create digital representations of the input data at higher abstraction levels Compare current representations with the definitions of objects/events stored in a DB Send representations and recognized objects/identified events to other parts of the architecture Store on a local DB representations and events generated by the subtasks
Information processing: context data Context Data Sensor Data (audio/video streaming, (... location GPS User profile (services most commonly ( preferences requested, Service usage (kind of subscription, ( fares active services, External context providers (sensors in the environment, weather (... situation 34
Information processing: content analysis Data Fusion of information coming from Context Source layer Different sub-task classes Representation tasks Context Recognition tasks Communication tasks 35
Information processing: content analysis Representation tasks information-processing sets of Instructions aimed at providing a representation of the data observed by a sensor High symbolic content Decomposition in sub-tasks 36
Information processing: content analysis Context Recognition tasks Object recognition Environment analysis Context representation Context reasoning Rules Ontologies Memory-based reasoning Embodied Cognition 37
Information processing: content analysis Communication tasks Data compression Connection to local networks ( bluetooth (wireless, IR, XML Tagging 38
Information processing: service integration Service integration User support Services on demand Context-based services 39
Mobile service model: a context aware example Context Aware Telecommunication Platform 40
Mobile service model: a context aware example Context Aware Telecommunication Platform Context Broker Aggregates, caches, context information Context Source Provides context updates from sensors Context Consumer Request context information through the context broker Context Provider Provides context on request using backend systems Content History Past events, functionality requests 41
Cognitive Telecommunication Systems Cognitive systems can improve the quality of telecommunication system services: Cooperative heterogeneous sensors fusion decision Multi mode user adapted functionalities ( indexing Intelligent storage (DB automatic semantic Easy multisensor autocalibration
Cognitive Telecommunication Systems Cognitive systems can improve the quality of telecommunication system services (cont.): Improved real-time scene understanding and data fusion tools Distribution of intelligence Distributed resources optimization Self-Learning Capabilities
Cognitive Telecommunication Systems Features: Scalability Architecture extendability adding nodes to various levels Mobility Nodes, sensors and control centers on mobile terminals Accessibility Signals and data stored in distributed DB for off-line analysis Quality of service improvement 44
Cognitive Telecommunication Systems Features (cont.): Reconfigurability Dynamic allocation of resources/functionalities Multi-user system Different data scope for different kind of users Privacy management Multi-sensor system Stability 45
Cognitive architecture SS: smart sensor SS CN: cognitive node SS SS CN CCC CC: cognitive control center CUT: cognitive user terminal HMI: human machine interface SS CN CUT HMI SS SS SS SS 46
Cognitive architecture Tree-like structure[1]: SS Nodes: CN Smart sensors Cognitive nodes Cognitive control centers Links: Heterogeneous communication channels SS CCC SS SS SS Each node has autonomous processing capabilities Distributed DB management 47
Smart sensors Which is the difference between a smart sensor and a normal sensor? A smart sensor has: its own internal cognitive cycle its own resident software agents 48
Smart sensors Smart sensors Heterogeneous sensing typologies: Location GPS, GSM signal User interaction Active application, idle/active status, phone alarm profile, charger status, and media capture Media Capture Camera/camcorder, microphone 49
Smart sensors Smart sensors Heterogeneous sensing typologies: Physical environment Optical markers, IR tags, temperature, pressure, humidity, accelerometer, gyroscope Communications environment Surrounding bluetooth/wireless devices 50
Smart sensors Smart sensors DECISION ACTION Data acquisition: communication from higher levels E ANALYSIS SENSING (... signals environment data (audio signals, audio signals, radio Data preprocessing: noise filtering, GPS sync, image processing (... balance (saturation, light adjustment, colour Data analysis: feature extraction, object recognition, signal context classification DECISION ACTION E ANALYSIS SENSING 51
Smart sensors Smart sensors DECISION ACTION Decision action to pursue: self decision: reconfiguration/optimization comm decision: communications to higher levels E ANALYSIS SENSING Parameters optimization camera focus, microphone noise threshold... Output to Cognitive node: objects or events detected ( tagging (e.g. via XML/OWL/RDL DECISION ACTION E ANALYSIS SENSING 52
Cognitive nodes A cognitive node takes as input the data representation from sensors level and through data fusion and object recognition produces a more abstract representation of the objects and events in the context. To each cognitive node can be linked various smart ( virtual sensors (physical or 53
Cognitive nodes Cognitive nodes DECISION ACTION Data acquisition: objects and events from smart sensors communications from higher levels ANALYSIS E SENSING Data fusion: combine data from multiple heterogeneous sensors and related information from associated databases data and object correlation DECISION ACTION E ANALYSIS SENSING Data analysis: object recognition 54
Cognitive nodes Cognitive nodes Decision action to pursue: self decision: reconfiguration/optimization comm decision: communications to lower/higher levels DECISION ACTION E ANALYSIS SENSING 55
Cognitive nodes Cognitive nodes Parameters optimization: memory allocation, computational resources.. algorithm optimization Output to Cognitive Control Center: events or scene description ( tagging (e.g. via XML/OWL/RDL Output to Smart Sensors: sensor activation/functionality request DECISION ACTION E ANALYSIS SENSING 56
Cognitive Control Center The CCC is the highest level of the architecture; its decisions can modify the functioning of the entire system (e.g. modifying parameters, adding or (... priorities removing functionalities, setting Data coming from cognitive nodes are again fused in order to obtain an overall description of the scene/environment User interface management 57
Cognitive Control Centers Cognitive control centers DECISION ACTION Data acquisition: scene descriptors from Cognitive Nodes ANALYSIS E SENSING Data fusion: combine objects/events detected by various cognitive nodes related information from associated databases Data analysis: scene/environment understanding DECISION ACTION E ANALYSIS SENSING 58
Cognitive Control Centers Cognitive control centers DECISION ACTION Decision action to pursue: self decision: reconfiguration/optimization comm decision: communications to lower levels E ANALYSIS SENSING Parameters optimization: memory allocation, computational resources.. algorithm optimization Output to User level: Human Machine Interface DECISION ACTION E Output to Cognitive Nodes: functionality request ANALYSIS SENSING 59
Concurrent agent computing model A possible SW implementation of the architecture previously described uses software agents Agents can be used to encapsulate existing software systems so as to resolve sub-problems and data integration, to represent physical resources such as sensors, interfaces, communication links, as well as planning, scheduling and execution algorithms. 60
Concurrent agent computing model Software agent: a special kind of software which can execute autonomously; once dispatched, it can perform data processing autonomously[2] Advantages: Data stays at local sites, processing task is moved Reduced network bandwidth requirement Better network scalability Extensibility Stability Smart Sensor resident agents 61
Concurrent agent computing model Agents are very useful for monitoring different data sources for specific data. Agents can be dispatched/activated/deactivated in remote locations to accomplish specific tasks.[3] No centralized system control structure, and no predefined agenda for the system execution The overall behaviour of the cognitive cycle emerges from the concurrent data processing of the single agents 62
Algorithms The SW agents previously described can be implemented to perform the various tasks needed in the cognitive architecture using cognitive algorithms such as: Data acquisition (processing dependent on the characteristic ( source of the data signal processing and conditioning standardize inputs extract key information feature extraction 63
Algorithms Data fusion data alignment association and correlation Classification/Estimation Object/event recognition Kalman filter Mean Shift Models for context-based decision Knowledge representation Core-self, proto-self, autobiographical memories Embodied cognition 64
Cognitive architecture Each level of the architecture has an internal cognitive cycle: sensing analysis decision action The entire system behaves according to a macrocognitive cycle. Human users can be described by CC too The whole couple user/cognitive system can be considered as a mirror interaction that involves two cognitive entities/cycles 65
Cognitive system example: virtual guide User-Guide interaction: cognitive cycle decomposition EXTERNAL INTERNAL STATE STATE Sensing Analysis & Repr. Decision Action & Comms SYSTEM Mirror representation X Action & Comms Decision Analysis & Repr. USER Sensing yes no no X X X EXTERNAL INTERNAL STATE Corso di Sistemi Cognitivi per Xle Telecomunicazioni Prof. C. Regazzoni X
Bibliography L. Nadel, Preface of the book Cognitive Systems: Information Processing Meets Brain Science, R. Morris, L. Tarassenko, M. Kenward Editors, Elsevier Academic Press, London, UK, 2006 A. Newell, 82 Expert Systems McCullock Rosenblatt Brooks Llinas 67
Bibliography [2] C. S. Regazzoni and A. Tesei, Distributed data fusion for real-time crowding estimation, Signal Process., vol. 53, pp. 47 63, 1996. [3] H.Qi, X.Wang et al., Multisensor Data Fusion in Distributed Sensor Networks Using Mobile Agents [4] P.Netinant Building agent-based intelligent concurrent systems 68