Knowledge ecosystems

Open problems

Even though progress in AI has been impressive, it remains largely unclear how AI can be created and utilized to support scientific knowledge creation, discourse and decision making most effectively. This is caused by the following fundamental problems that need to be addressed:

1. Lack of a big-picture conceptualization, analysis and understanding of which intelligence tasks exist that make up scientific knowledge creation processes, how they interrelate, which tasks are most important and how they can be formulated and operationalized for AI applications.

2. Lack of a good understanding of how well current AI models address, or fail to address these tasks. Many existing benchmarks only insufficiently capture capabilities required in real environments, and for many potential applications, benchmarks are still missing altogether.

3. Lack of a good understanding of what the main barriers for the effective development and deployment of AI for each process in scientific knowledge creation are, such as missing datasets, missing benchmarks, lacking AI capabilities and technology readiness, lacking trustworthiness and explainability, lacking AI technology diffusion or missing/misaligned incentives.

4. Finally, caused by the above problems: Lack of a strategy for developing and deploying AI for scientific knowledge creation in the most effective and beneficial manner.

The scientific knowledge creation ecosystem

We conceptualize the core processes of the scientific knowledge creation ecosystem as a multifaceted, interrelated ecosystem.

A model of the core processes of the scientific knowledge ecosystem.

Examples of processes of the scientific knowledge creation ecosystem and the AI tasks that can potentially support them are:

Searching and querying knowledge sources, e.g. assistance with query formulation, AI-augmented information retrieval, extractive and abstractive question answering.
Curating knowledge, e.g. relevant literature identification, document classification, named entity recognition, relation extraction, assistance with data entry, automated knowledge base creation.
Linking and integrating knowledge, e.g. semantic sentence similarity estimation, ontology alignment, knowledge base alignment, knowledge base completion.
Analyzing and understanding knowledge sources, e.g. automated text summarization, visualization, generation of explanations.
Learning and teaching, e.g. in human sense: creating learning material, AI-assisted interactive learning experiences; in statistical/AI sense: creating useful and predictive models.
Inference and hypothesis generation, e.g. link prediction in knowledge graphs, automated hypothesis generation, natural language inference.
Discourse, reviewing, consensus finding, e.g. sentiment analysis, discourse summarization, expert finding, related literature recommendation, social media for scientific discourse, translation.
Acting: Conducting studies, making decisions, e.g. assistance with study processes and logistics, data gathering/documentation/processing, guideline adherence, digital decision support tools in clinical studies.
Producing, formalizing and publishing knowledge and ideas derived from studies, e.g. scientific writing assistance, creating structured publications or knowledge bases, adherence to FAIR guidelines.

Additionally, there are ‘meta-processes’ dealing with the scientific knowledge ecosystem itself, such as: Benchmarking and evaluating methods and processes, and continuous updating of goals and incentives of the system as a whole.

Goals

1. Modeling and operationalizing

We aim to achieve a novel and more complete conceptualization of the problem space by developing a comprehensive formal model of human and artificial intelligence tasks, benchmarks, performance measures, datasets and the current state-of-the-art in AI.

Questions we aim to address:

• What are the core processes of the scientific knowledge ecosystem and their sub-processes?
• What relevant data are available, and what processes are required to generate more relevant data?
• What existing relevant data sources are there?
• What are the different representations of knowledge in these data sources?
• What are the current processes that generate these data and how can AI improve generation of further data?
• Which relevant benchmarks are available, and what processes are required to create more and better relevant benchmarks?
• Which benchmarks exist and which are still needed for measuring and steering AI capabilities?
• Which statistical measures are most appropriate for quantification?

🠆 Read more about our work on mapping machine intelligence.

2. Identifying the actors in the scientific knowledge ecosystem

Problems and barriers are often not purely of a technical nature. An understanding of the driving actors in the scientific knowledge ecosystem, their goals and incentives is important. In many cases, incentives are missing or misaligned, or the incentives of different actors are partially opposing. We aim to develop an understanding of the actors in the scientific knowledge ecosystem and devise strategies to create synergies.

Questions we aim to address:

• What are the main types of actors (e.g. professions, types of organisations) of the knowledge creation system and how are they interacting with each other?
• What are their explicit and implicit goals and incentives?
• How to create synergies between actors?

3. Effecting change

We aim to to devise strategies to promote change to maximize speed of knowledge creation and innovation, while minimizing risk and unwanted side-effects.

Questions we aim to address:

• What are the processes of the system with good potential and tractability of change?
• What risks and potential unwanted side-effects of change are there and how can they be mitigated?
• How can impactful and beneficial changes be implemented?