Developing an Action Plan to Implement and Evaluate Obesity Prevention in Early Care and Education

Action Plan - Man on Arrow Over WordsAction plans are important because they keep you on track and focused by setting objectives and timelines, provide guidelines for achieving objectives, help you monitor your progress and successes, and give you a clear vision of what you are going to do and what outcomes to expect.The information, worksheets, and templates provided in the Quick Start Action Guide are intended as a road map for action.

An action plan firmly states goals, measurable objectives, and time-phased action steps. It also identifies resources and responsible individuals, groups, or organizations and describes how to evaluate the activity. It is important to include a well-developed evaluation component in order to further the evidence base for the spectrum of opportunities. Developing an action plan that can be shared with stakeholders is a helpful way to solidify activities and identify partners, financial resources, and other inputs that are important to initiating change in the ECE arena.

Many states and communities have a variety of existing plans related to both the ECE system and obesity prevention and health promotion.  Upon drafting a plan specific to obesity prevention in ECE, you can determine whether it is more valuable to finalize it as a stand-alone plan or to integrate its components into preexisting plans, or both.
.

Action Plan Development

The first step in developing an action plan is to identify a goal or set of goals to achieve. This may be done though a goal-setting process with other stakeholders, or the goal may originate from a legislative directive or federal standard. Either way, a goal is a statement that explains, in general terms, what you wish to accomplish, and expresses long-range desired outcomes. Typically, goals are broad, general statements. An example of a goal might be “to achieve recommended standards of physical activity in ECE settings throughout the state.”

Once a goal or goals have been identified, specific objectives can be delineated. Objectives provide specific, measurable actions by which the goal can be accomplished. Objectives define for stakeholders and partners the results expected from implementing the action plan. An example of an objective might be “to increase by 50% within 1 year the number of ECE facilities that have a policy requiring children to be provided at least 60 minutes of physical activity daily while in care.”

For objectives to be effective components of an action plan they should be developed using the “SMART” framework. SMART stands for1:

Specific

Measurable

Attainable/Achievable

Relevant

Time bound

SMART objectives answer the question “WHO is going to do WHAT, WHEN, and TO WHAT EXTENT?” When an action plan includes well-written objectives, it is easier to develop time-phased action steps and identify the resources and/or partnerships that are crucial to the implementation process. SMART objectives also become the basis for solid program evaluation.

Once goals and objectives have been established, use the Action Plan Worksheet to work through the specific details including action steps; materials, resources, and personnel needed/involved; time frames; evaluation methods; and other important information. A worksheet for planning facility-level interventions is also included as an example of how you might think about planning a specific type of activity.
.

Expressing your Action Plan as a Logic Model

With a goal and SMART objectives in place, the components of your action plan can be placed into a logic model that visually depicts your inputs, activities, outputs, and outcomes (see example logic model). Creating a logic model has many benefits including setting partners’ expectations, monitoring progress, and identifying evaluation questions. The logic model identifies short-term, intermediate, and long-term outcomes and shows why activities are expected to lead to desired outcomes.

A logic model contains four core components:

  • Inputs: resources, contributions, investments
  • Activities: actions, events
  • Outputs: immediate products (trained staff, meeting attendees)
  • Outcomes: changes related to objectives
    .

Evaluation Planning

To determine the effectiveness of policy or practice change, one must document and measure both its implementation and its outcomes, which is done through program evaluation. Program evaluation is “the systematic collection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, improve program effectiveness, and/or inform decisions about future program development.”2 Evaluation should be practical and feasible and must be conducted within the confines of resources, time, and political context. Moreover, evaluation should serve a useful purpose, be conducted in an ethical manner, and produce accurate findings. Evaluation findings should be used to make decisions about policy or program implementation and to improve effectiveness.  See the Introduction to Program Evaluation for Comprehensive Tobacco Control Programs  and   for more information about program evaluation.
.

Evaluation planning begins with clearly defining the purpose of the evaluation and identifying who will use the evaluation findings and to what end. This step can help to generate the evaluation questions that are the foundation of a good evaluation.  Evaluation can serve different purposes, such as:

  • To determine progress toward achieving outcomes;
  • To improve programs and policies;
  • To leverage resources and support; or
  • To provide accountability to funders.
    .

Users and uses of evaluation findings may include:

  • Partnership leadership can use findings to provide accountability and show progress toward achieving outcomes to inform the work of the partnership. Findings can also be used to determine areas in need of improvement to facilitate the accomplishment of activities.
  • Management staff can use evaluation data to help improve the implementation of the action plan and the engagement of partners in the implementation of the plan.
  • Partnership members can use the evaluation findings to support their participation in the partnership, to support implementation of the plan, and to advocate on behalf of the action plan.
  • Funders may use findings to support continued funding, to leverage additional funding, and to identify successful strategies to share with other grantees.
  • People affected by action plan activities may use evaluation findings to support implementation of strategies and promote awareness of the importance of improved nutrition, breastfeeding, physical activity, and screen time practices in the ECE setting.
    .

It is important to consider how the evaluation will answer stakeholder questions and provide continuous feedback at all levels of action plan implementation.  Once you have defined your purpose, use, and users of your evaluation, you can begin to develop your evaluation plan.

.

The evaluation plan will help develop an overall picture of evaluation activities so that required staff time and resources can be identified. This plan should be based on the policy or program objectives stated in your action plan and provide an approach to assess the extent to which those objectives have been achieved.  To view a sample logic model and potential evaluation questions for the ECE action plan click here.  See the evaluation guide, Developing an Evaluation Plan for more information.

.

What is Different about Evaluating Policy?

A common question asked about evaluation is whether evaluating policy is different from evaluating programs and interventions. The evaluation design is the same. The difference is that policy change is a longer-term effort and often requires 3 to 10 years to achieve changes in population-based practices, behaviors, and health outcomes compared with interventions and program outcomes that can be achieved, and thus evaluated, in a much shorter time period.

The stage of development of your policy intervention is critical for determining the appropriate focus for the evaluation. For instance, evaluation of a ECE nutrition and physical activity policy in the development stage should focus on deciding the standards to include in the policy, acceptance and feasibility of the standards by child care providers and managers, infrastructure to support implementation of the policy, and approval and endorsement of the policy. Evaluation of the implementation of the policy should focus more on the inputs or resources that have been allocated, the activities that are being implemented, and achievement of expected outcomes.

The policy development process itself is unique. State-level policy activity typically develops along a continuum that is impacted by three processes: problem, proposals, and politics. Problem refers to the process of persuading policy makers to pay attention to one problem over others. Proposals represent the process by which policy solutions are generated, discussed, and agreed upon. The policy proposals are influenced by the organizational culture, including willingness to make a change and the availability of resources. This process can often present challenges among a diverse group of partners, especially in reaching agreement on which approaches are most appropriate for addressing a particular health problem. Politics are the factors that influence political agendas, such as changes in elected officials, political climate, and support or opposition by key leaders, policy makers, and the public. A “policy window” opens that increases the chances of passing a bill when at least two of these three processes (problem, proposal, or politics) occur at the same time. If all three processes are linked together the chances of passing a bill substantially increases.3

When thinking about evaluation, however, it is important to note that sometimes the outcome of a bill being passed or a policy being adopted does not happen for reasons unrelated to the quality of the policy or its development process. For this reason when evaluating policy, benchmarks should be set in advance that identify the incremental milestones in achieving policy goals that define state-level progress, including the creation of awareness and the enhancement of capacity for future policy development, adoption, implementation, and enforcement.

For sample logic models and potential evaluation questions for policy development and implementation click here.

 

References

  1. Centers for Disease Control and Prevention, Division of Heart Disease and Stroke Prevention. Evaluation guide: Writing SMART objectives. Retrieved from http://www.cdc.gov/dhdsp/programs/nhdsp_program/evaluation_guides/docs/smart_objectives.pdf.
  2. Patton MQ. Utilization Focused Evaluation: The New Century Text. Thousand Oaks, CA: Sage; 1997.
  3. Kindgon JW. Agendas, Alternatives, and Public Policies, 2nd ed. New York: Longman; 1995.
  • Is your policy or program making a difference?
  • Is your policy or program effective in improving nutrition, breastfeeding, physical activity, and screen time practices in the ECE setting?
  • Can your policy or program be improved? How, in what ways?
  • What exactly is your policy or program achieving?
  • Is your policy or program accomplishing what it was intended to accomplish?
  • Was your policy or program implemented as planned?
  • Are you using resources efficiently and effectively?
  • What does your target population or audience think of your policy or program?
  1. Develop evaluation questions (What do you want to know?)
  2. Determine indicators (What will you measure? What type of data will you need to answer the evaluation question?)
  3. Identify data sources (Where can you find these data?)
  4. Determine the data collection method (How will you gather the data?)
  5. Specify the time frame for data collection (When will you collect the data?)
  6. Plan the data analysis (How will data be analyzed and interpreted?)
  7. Communicate results (With whom and how will results be shared?)
  8. Designate responsibility (Who will oversee the completion of this evaluation?)