How to Choose a Good Scientific Problem

how-to-choose-a-good-scientific-problemChoosing a good scientific problem is essential for being a good scientist. But what is a good problem, and how to choose one? The subject is not usually discussed explicitly within our profession. Scientists are expected to be smart enough to figure it out on their own and through observation of their teachers. This lack of explicit discussion leaves a vacuum that can lead to approaches such as: choose problems that can give results that merit publication in valued journals, resulting in a job and tenure.

The premise of this essay is that a fuller discussion of our topic, including its subjective and emotional aspects, can enrich our science, and our well-being. A good choice means that you can competently discover new knowledge which you find fascinating and which allows self-expression.

We will discuss simple principles of choosing scientific problems that have helped me, my students and many fellow scientists. These principles might form a basis for teaching this subject generally to scientists.

.

Starting point: choosing a scientific problem is an act of nurturing

What is the goal of starting a lab? It is sometimes easy to pick up a default value, common in current culture, such as “the goal of my lab is to publish the maximum number of papers of the highest quality.”

However, in this essay, we will frame the goal differently: “A lab is a nurturing environment that aims to maximize the potential of students as scientists and as human beings.”

Choices such as these are crucial. From values, even if they are not consciously stated, flow all of the decisions made in the lab, big and small: How the lab looks, when students can take a vacation, and – as we will now discuss – what problems to choose. Within the nurturing lab, we aim to choose a problem for our students (and for ourselves) in order to foster growth and self-motivated research.

.

The two dimensions of scientific problem choice:

To choose a scientific problem, let us begin with a simple graph, as a starting point for discussion. We will compare problems by imagining two axes. The first is feasibility, that is — whether a problem is hard or easy, in units such as the expected time to complete the project. This axis is a function of the skills of the researchers and of the technology in the lab. It is important to remember that problems that are easy on paper are often hard in reality, and that problems that are hard on paper are nearly impossible in reality.

The second axis is interest: the increase in knowledge expected from the project. We generally value science that ventures deep into unknown waters. Problems can be ranked in terms of the distance from the known shores, by the amount in which they increase verifiable knowledge. We will call this the interest of the problem.

In a forthcoming section, we will discuss the subjective nature of the interest axis. But first, let us first consider aspects of problem choice using our diagram.

Looking at the range of problems in this two dimensional space, one sees that many projects in current research are of the easy-but-not-too-interesting variety, also known as ‘low hanging fruit’. Many other projects in science today are unfortunately both difficult and have low interest, partially stemming from a view that hard equals good. A few problems are grand challenges: tough problems with the potential to considerably advance understanding. But most often we would like problems in the top right quadrant, both feasible and with high interest, likely to extend our knowledge significantly.

The diagram suggests a way to choose between problems, using the Pareto front principle of optimization theory. If problem A is better on both axes than problem B, one can erase B from the diagram. Applying this criterion to all problems, one is left only with problems for which there are no problems clearly better in both feasibility and interest. These remaining problems are on the Pareto front.

To decide which scientific problem to select along the front depends on how we weigh the two axes. For example, a beginning graduate student needs a problem that is easy: positive feedback can thus be rapidly provided, bolstering confidence. These problems are on the bottom right of the Pareto front. The second problem in graduate school can move up the interest axis. Postdocs need projects in the top right quadrant, since time is limited. Beginning PIs, that need to select a field on which to spend at least ten years and with which to train students, may seek a grand challenge that can be divided into many good, smaller projects. Thus the optimal problems move along the Pareto front as a function of the life stages of the scientist.

Take your time:

A common mistake made in choosing a scientific problem is taking the first problem that comes to mind. Since a typical project takes at least two years even it if seems to be doable in two months, rapid choice leads to much frustration and bitterness in our profession. It takes time to find a good problem, and every week spent in choosing one can save months or years later on.

In my lab we have a golden rule for new students and postdocs: do not commit to a scientific problem before three months have elapsed. In these three months the new student or postdoc reads, discusses, plans. The state of mind is focused on being rather than doing. The temptation to start working arises, but a rule is a rule. After three months (or more), a celebration marks the beginning of the research phase – with a well planned project.

Taking time is not always easy. One must be supported to resist the urge “Oh we must produce, let’s not waste time and start working”. Taking time can be especially difficult when funding is insufficient and grant deadlines approach. In such difficult situations, nurturing is not enough, and you need to find support and do all you can to get into a better situation. Even so, for many of us dealing with the difficulties of running a lab, taking time to choose problems can make a huge difference.

.

_____________________________________________________________________________________

Uri is a Professor in the Department of Molecular Cell Biology at the Weizmann Institute of Science. He is currently on sabbatical in the Department of Systems Biology at Harvard Medical School.  The full post can be found in the “Materials for Nurturing Scientists” section of Uri’s website.

.

.

5 comments so far. Join The Discussion

  1. [email protected]

    wrote on August 3, 2009 at 5:12 pm

    I've literally sat and overlayed my ideas onto a similar type of graph as seen above- trying to figure out which one might have the biggest impact with the most straightforward experiments. It's easy to get caught up with the size of the potential discovery without really thinking about the reality of how to get there. It's also a great way to prioritize your "main" project and your "side" projects.

  2. The importance of side projects | BenchFly Blog

    wrote on August 13, 2009 at 1:41 am

    […] research project, Uri Alon provided an excellent strategy that balances risk and reward (see How to choose a good scientific problem).  But as we all know, there are no guarantees in science.  To minimize the chance of spending […]

  3. BenchFly's Guide to Year 2 of Graduate School | BenchFly Blog

    wrote on August 31, 2009 at 11:53 pm

    […] the post How to choose a good scientific problem by Uri Alon.  Great […]

  4. Natalie Sashkin Goldberg

    wrote on December 2, 2010 at 6:33 pm

    I have been advised to "go big" on a second year graduate project (in terms of knowledge gain and risk factor, not labor). Because the sway of results don't matter, if they are null you're not damaged and if they're great you have preliminary data for a thesis project.

    Thoughts from the veterans?

  5. larry

    wrote on December 3, 2010 at 6:53 am

    There aren't any easy answers to going after a big project in the second year of grad school. If you've developed the experimental skills and have a good working knowledge of the literature underlying the project, it's a very good time to take a risk. You're early in your career and you're about to finish your coursework and prelims so you'll be in a position to attack the problem with time and enthusiasm. You've got to be prepared to be flexible and learn new techniques and concepts because you're headed into uncharted waters. But that can be really energizing and almost everything you learn will be new, which is the point of research. So there's a high excitement quotient from swinging for the fences early in grad school. However, you've got to be prepared to recognize when you get to a point where your idea was either wrong or just isn't going to work. Knowing when to cut bait on a project is really difficult and requires close interactions and intense discussions with your advisor. The cut bait decision is always difficult and the sexier the idea, the harder it is to walk away. I've had ideas that just HAD to work but didn't and it's really painful to watch a talented grad student get ground up trying make something work that probably isn't going to. So I'd say take a risk and attack the problem aggressively. As you're working on plan A, though, be on the lookout for components that would make acceptable projects for plan B to get you a degree if plan A doesn't pan out. And maintain good and regular communications with your advisor on your progress.

Leave a comment

will not be published