knowledge

Science

Resilience & Robustness

Resilience is the capacity of a system to maintain functionality in the face of some alteration within the system’s environment. All systems exist within an environment and are, to a certain extent, dependent upon a specific range of input values from that environment. The system has a set of parameters to these inputs within which it can maintain its structure and functionality, but outside of these critical parameters the system will become degraded to a lower level of integration or functionality.

Resilience and robustness can then be defined by this set of parameters. The lower the system’s dependency upon its environment and the broader this range of input values that the system can operate within, the more robust it can be said to be.

There are fundamentally just two ways - mechanisms for resilience - for a system to maintain its integrity given some perturbation. It can resist this change or adapt to it.

Resist - a tree developing a sturdy trunk to withstand wind
Adapt - a tree bending in response to the wind

Complexity

Complexity arises in highly evolved biological and technological systems primarily to provide mechanism to create robustness.

The connection between advanced technology and biology is neither superficial nor accidental, and much can be learned from comparing and contrasting the organizational principles underlying complex biological and technological systems.

The universal system requirements to be efficient, adaptive, evolvable, and robust to perturbations in their environment and component parts.

A system can have a property that is robust to one set of perturbations and yet fragile for [a different property] and/or perturbation.

Complexity is most succinctly discussed in terms of functionality and its robustness. Specifically, we argue that complexity in highly organized systems arises primarily from design strategies intended to create robustness.

Building functional redundancy of critical components or subsystems is a popular and well-studied approach to robustness, but redundancy per se is a limited and blunt tool.

How To Spot Bad Science

Science is not some big immovable mass. It is not infallible. It does not pretend to be able to explain everything or to know everything. Furthermore, there is no such thing as “alternative” science. Science does involve mistakes. But we have yet to find a system of inquiry capable of achieving what it does: move us closer and closer to truths that improve our lives and understanding of the universe.

Good science is science that adheres to the scientific method, a systematic method of inquiry involving making a hypothesis based on existing knowledge, gathering evidence to test if it is correct, then either disproving or building support for the hypothesis. It takes many repetitions of applying this method to build reasonable support for a hypothesis.

No journal is perfect. Even the most respected journals make mistakes and publish low-quality work sometimes. However, anything that is not published research or based on published research in a journal is not worth consideration. Not as science. A blog post saying green smoothies cured someone’s eczema is not comparable to a published study. The barrier is too low. If someone cared enough about using a hypothesis or “finding” to improve the world and educate others, they would make the effort to get it published. The system may be imperfect, but reputable researchers will generally make the effort to play within it to get their work noticed and respected.

Meta-analyses, which analyze the combined results of many studies on the same topic, are often far more useful to the public than individual studies. Scientists are humans and they all make mistakes. Looking at a collective body of work helps smooth out any problems.

Questioning the existing body of research is not inherently bad science or pseudoscience. Doing so without a remarkable amount of evidence is.

The more outlandish the claim, the less likely it is to be true. Occam’s razor teaches us that the simplest explanation with the fewest inherent assumptions is most likely to be true. This is a useful heuristic for evaluating potential magic bullets.

Researchers do have to get funding from somewhere, so this does not automatically make a study bad science. But research without conflicts of interest is more likely to be good science.

In the vast majority of cases, a single study is a starting point, not proof of anything. The results could be random chance, or the result of bias, or even outright fraud. Only once other researchers replicate the results can we consider a study persuasive. The more replications, the more reliable the results are. If attempts at replication fail, this can be a sign the original research was biased or incorrect.

It’s not that bad presentation makes something bad science, but the way a paper looks can be a useful heuristic for assessing its quality.

Bad science is a flawed version of good science, with the potential for improvement. It follows the scientific method, only with errors or biases.

Pseudoscience has no basis in the scientific method. It does not attempt to follow standard procedures for gathering evidence. The claims involved may be impossible to disprove.

Good science is science that adheres to the scientific method, a systematic method of inquiry involving making a hypothesis based on existing knowledge, gathering evidence to test if it is correct, then either disproving or building support for the hypothesis.

Science is about evidence, not proof. And evidence can always be discredited.

In science, if it seems too good to be true, it most likely is.