Human infrastructure - build it bottom-up

An article by Procter et al "Fostering the human infrastructure of e-research" (2013, restricted access) discusses the challenges of embedding computing resources and systems into research. E-infrastructures (aka cyberinfrastructures in the US) are defined as digital information and communication technologies (ICTs) that can provide fast and scalable access to remote resources and increase discovery and innovation. Human infrastructure is arrangements of actors and organizations that make computer-research systems work. It is often acknowledged that human infrastructure is neglected compared to the investments in the technical infrastructure. And that's where the problem is. Cyberinfrastructures don't work without human adoption and use. As authors of the article state:

"So far, despite substantial investment, the desired transformative impact has yet to be achieved."

The article describes the Enabling Wider Uptake of e-Infrastructure Services project (ENGAGE/e-Uptake) that was designed to identify inhibitors and enablers of the adoption of e-Infrastructure services. The identification is based on interviews with ~50 researchers from higher education institutions and ~50 "intermediaries", or technical specialists who support researchers in their use of ICTs.

Findings

Obstacles in e-Infrastructure adoption and use:

• Lack of training - many researchers hear about research computing services, but they often don't know about the nature of the services and the benefits of using them.
• Lack of local research support - support is often basic, limited, fragmented and difficult to access.
• Poor project management - projects that involve technical and research personnel have their own managing needs, i.e., the need to manage collaborations between people with their own research agendas and temporarily aligned interests. Managers who don't have such skills may make biased decisions and favor one type of team members over others.

Conclusions

• There is complexity in divisions of labor and in organizational structures that may be historical. In cyberinfrastructure projects we may need more flexible and flatter approaches.
• More teaching and training is needed - not only teaching of e-Research methods in classes, but also lifecycle outreach from the collaborative formation of projects through the acquisition of skills and the appropriation of technologies to the dissemination of experiences back into the community (see, for example, eIUS project for a collection of use cases and tools used in them).
• User engagement can take a form of relying on "hybrids", i.e., people with both technical and domain expertise, or a form of co-locating technical experts and users throughout projects. More research is needed into how to do that plus how to leverage community engagement.
• New practices must be embraced not only by researchers, but by the organizations within which researchers work.

The article reinforces the idea that by default software and computing tools are hard to learn and use. Why is that? A common argument is that complex problems require complex solutions. Doesn't a simple fix sometimes work better? Or, perhaps, it's ok to have complex solutions, but they arise from a number of simple solutions combined and overlapped. I wonder whether we should start with building simple local systems ("recognized routes" or local roads) rather than large and multi-purpose systems (interstate highways, to continue the infrastructure metaphor). Once local needs are met and served well, we can move into connecting local systems (i.e., building bridges, gateways, etc.). It circles back to the investment in human infrastructure and bottom-up rather than top-down approaches.

NIH report: Big data recommendations based on small data?

I've been browsing slides from the last BRDI Symposium, "Finding the Needle in the Haystack: A Symposium on Strategies for Discovering Research Data Online", and found a report for the National Institutes of Health about the management and analysis of large biomedical research data (pdf available here).

It is an interesting report that provides a lot of details about data and technologies in biomedical research as well as about existing efforts in data sharing. The recommendations make sense, since they follow most of the recommendations with regard to research data - more money, more policy, more training:

• Promote data sharing by establishing a minimal metadata framework for data sharing, creating catalogs and tools and enhancing data sharing policy for NIH-funded research.
• Support the development and dissemination of informatics methods and applications by funding software development.
• Train the workforce in quantitative sciences by funding quantitative training initiatives and enhancing review expertise in quantitative methods of bioinformatics and biostatistics.

Even more interesting is what evidence is provided to support these recommendations. The report is based on a relatively small literature corpus (~25 citations plus footnotes) and on the analysis of comments that were solicited via an NIH request for information on management, integration, and analysis of large biomedical datasets. Overall, 50 respondents replied and made 244 suggestions. Is it enough data to make recommendations for NIH? If we begin with the assumption that more support for large datasets and biomedical computations is needed (which seems to be the case with this report), then there is almost no need to analyze costs and benefits of data sharing, the role of large datasets in providing solutions for biomedical problems, and so on.

Max Weber on ethical neutrality in the social sciences

Finally finished a piece by Max Weber "The meaning of 'ethical neutrality' in sociology and economics" (Methodology of social sciences, 2011, google books link).

In this piece Weber asks whether the social sciences can be ethically neutral and what it means in terms of their research questions and methods. He addresses this issue by distinguishing between value-judgments and factual assertions. Value-judgments are evaluations of phenomena that can be satisfactory or unsatisfactory (positive or negative). They are derived from ethical principles or cultural ideals, which are subjective and therefore cannot be discussed scientifically. Factual statements are logically deducible and empirically observable. While the distinction between empirical statements and value-judgments is difficult to make, it is important according to Weber to keep making this distinction to maintain rigor in the social sciences. Avoiding taking a moral stand as part of one's research is what makes the social sciences science.

Weber's position is that education (and lectures as its ultimate manifestation in his times) should not be based on value-judgments. Students attend education institutions to cultivate their capacities for observation and reasoning, and a certain body of factual information. Evaluations, which cannot be contested in a lecture hall, should take place somewhere else. However, Weber writes that university decision-makers can decide which path to choose: to include value-judgments in education or not. It depends on whether they believe that education is about molding human beings and developing their political, ethical, and cultural attitudes, or whether it should focus on specialized training.

The methodological question in empirical sciences is not how to avoid value-judgments, but how to distinguish between them and empirical propositions and use both accordingly. Science can ask questions about things which convention makes self-evident. Evaluations (value-judgments) often seem self-evident. They can be examined by empirical sciences with respect to the conditions of their emergence and existence. This leads to an “understanding", i.e., a greater awareness of the issues and reasons for persistence of norms and opinions as well as conflicts. Empirical sciences can help to understand the means, the repercussions, and the conditioned competition of various evaluations, but choices between means, consequences and ultimately evaluations are matters of choice and compromise.

There is no (rational or empirical) scientific procedure of any kind whatsoever which can provide us with a decision here. The social sciences, which are strictly empirical sciences, are the least fitted to presume to save the individual the difficulty of making a choice, and they should therefore not create the impression that they can do so. - p. 19

One of the tasks of an empirically neutral social science is to analyze standpoints and reduce them to rational, internally consistent forms and investigate the pre-conditions of their existence and their implications. It can be done by using theoretical constructs, ideal types, which are pure fiction and should be used as such. Ideal types, or rationally correct and consistent Utopian constructions of patterns or behaviors are useful in comparing them with empirical reality in order to establish its divergences or similarities and to understand or explain them causally. Ideal types should not be used for establishing moral imperatives.

In theory, Weber's approach makes sense. Especially, when he talks about the danger of presenting value-judgments as factual statements and making them imperatives. It's obvious that mixing evaluations with facts makes a bad science. But what happens when we make a conscious choice to remain ethically neutral when studying sensitive issues or vulnerable populations? Also, if we become aware of means and repercussions of evaluations, why doesn't it help us to make better choices?

Case study research

A short note on case study research (from “Case study research: Design and methods” by R. K. Yin):

Case studies are good for “how” and “why” questions, when the researcher has little control over events and the events happen in real-life context. Case study approach allows to retain the holistic character of a complex situation.

Common tendency among different types of case studies: it seeks to illuminate a decision (or a set of decisions), e.g., why they were made, how they were implemented, and with what result. Other tendencies: focus on communities, processes, events.

Definition:
a case study is an empirical inquiry that investigates a phenomenon within a real-life context, especially when the boundaries between phenomenon and context are not clear. Case studies rely on multiple sources of evidence with data needing to converge in a triangulating fashion. Case studies benefit from the prior development of theoretical propositions to guide data collection and analysis.

Data for case studies comes from many sources of evidence. Most important are documents, archival records, interviews, direct observation (site visits), participant observation and physical artifacts.

Three principles of data collection:

• using multiple sources of evidence (triangulation of data, investigators, perspectives, methods)
• creating a case study database (to store all the materials for many case studies in an organized manner)
• maintaining a chain of evidence (keeping information on how the study moved from one stage to the other, e.g., from study questions to study protocols)