Key Findings from the ChildTech R01 Exploratory Longitudinal Pilot Study

 

​The empirical case was situated in Hungary and was based on the ChildTech Summer 2026 email campaign, distributed to a voluntarily subscribed professional audience interested in or directly involved in the Hungarian ECEC field. Most subscribers had entered through communication channels associated with a professional community of preschool educators, although recipient-level occupational data were not available.

​

Digital communication platforms generate large volumes of routinely recorded data. Email systems report opens, clicks, unsubscribes, bounces and other interaction metrics as part of everyday campaign management. For professional communication in early childhood education and care (ECEC), these data may appear to offer a readily available basis for longitudinal research.

 

Yet operational platform data do not automatically constitute research data.

 

The central question examined in ChildTech Report 01 (R01) was therefore not whether one email campaign performed better than another. Instead, the study asked under what conditions routinely generated email-platform data can be transformed into a controlled, interpretable and auditable longitudinal research structure.

R01 was designed as an exploratory longitudinal pilot study based on three consecutive waves of the ChildTech Summer 2026 professional email campaign. Its primary pilot question concerned measurement feasibility: what methodological limitations and design requirements emerge when operational email analytics are reused for longitudinal research?

 

The empirical case

 

The empirical dataset consisted of three campaign waves—SU1, SU2 and SU3—each implemented through four technical sending units. This produced 12 sending units in total.

The historical operational protocol included five repeated measurement points, M1–M5, for every sending unit. These generated 60 protocol observations. A further 12 observations were collected later at a common calendar time as an extended FOLLOWUP layer.

 

The resulting analytical dataset therefore contained 72 aggregated observations.

An important distinction was maintained throughout the study: FOLLOWUP was not treated as M6. It represented a separate observational layer rather than a sixth point in the original protocol. The available dataset was aggregated at sending-unit level and did not contain recipient-level longitudinal event sequences. The study therefore did not make recipient-level inferences and did not apply survival or hazard models.

 

Response activity develops over time

 

One of the clearest empirical findings concerned temporality.

A substantial proportion of the Open Rate eventually observed at M5 had already accumulated by the first recorded observation. Across the 12 sending units, Response_Accumulation_OR (RA_OR) ranged from 58.3% to 85.0%.

 

At first sight, this might suggest that the first observation provides a comparable view of early response across campaigns. The historical measurement structure showed why that interpretation would be misleading.

M1 did not occur at the same elapsed time after sending for every sending unit. The first measurement was recorded between approximately 1.93 and 9.97 hours after T0, while M5 occurred between approximately 52.98 and 159.18 hours after sending.

 

Consequently, a label such as “M1” identified the order of measurement, not a common temporal horizon. For longitudinal analysis, R01 therefore introduced Measurement_Lag_Hours, defined as the actual number of hours elapsed between an observation and the relevant sending unit’s own T0.

This distinction is methodologically fundamental: Measurement_No identifies where an observation stands in the operational measurement sequence.

 

Measurement_Lag_Hours identifies where that observation stands in actual elapsed time. The two variables are related, but they are not interchangeable.

 

Similar campaign-level Open Rates, heterogeneous response profiles

 

At M5, the recipient-weighted descriptive Open Rates for the three campaign waves were relatively close:

• SU1: 41.30%

• SU2: 40.77%

• SU3: 40.88%

The corresponding recipient-weighted descriptive Click Rates were:

• SU1: 1.27%

• SU2: 1.40%

• SU3: 1.57%

 

Because only three actual campaign waves were observed, and because the A–D sending units were technical parts of the same campaign waves rather than independent campaign replications, the study did not interpret this sequence as evidence of a temporal trend.

 

The more informative pattern appeared at sending-unit level.

 

Open Rate, Click Rate and Clicks per Unique Open did not show a single common ordering. For example, one sending unit could have a comparatively high Open Rate while showing little or no click activity, whereas another could display a more moderate Open Rate but substantially higher click-related activity.

This heterogeneity supports a simple but important analytical rule: platform-recorded response dimensions should not be collapsed into a single undifferentiated engagement score without justification. Opening, clicking and unsubscribing represent different types of platform-recorded events and should be analysed separately.

 

A platform-recorded open is not the same as reading

 

A second major methodological issue concerns measurement validity. Email platforms use technical mechanisms to detect events. Mailchimp open tracking, for example, depends on the loading of a small image element. Recorded events may therefore be affected by technical processes such as image loading, privacy-related preloading, automated systems or bots.

 

For this reason, R01 treats Open Rate as Mailchimp-recorded Open Rate.

It is not interpreted as a direct measure of:

• reading,

• psychological attention,

• comprehension,

• learning,

• or professional use of content.

 

The same inferential discipline applies to clicks. A recorded click is a platform event and cannot automatically be equated with intentional human action.

 

To control this problem, the study used a four-stage interpretive structure: platform definition - technical measurement process - possible behavioural interpretation - inferential boundary

 

This control does not invalidate platform metrics. Rather, it defines what can legitimately be concluded from them. Operational analytics remain useful research inputs—but only when their technological mediation is explicitly acknowledged.

 

Protocol-final is not necessarily response-final

 

The extended FOLLOWUP observations revealed another temporal issue.

Open Rate increased between M5 and FOLLOWUP in 10 of the 12 sending units, showing that additional platform-recorded activity could continue after the final scheduled protocol measurement. However, the time between M5 and FOLLOWUP was highly unequal across sending units, ranging from approximately 6 to 798 hours. The raw change between M5 and FOLLOWUP therefore cannot be interpreted as a clean comparison of long-tail response strength.

 

R01 distinguishes two concepts:

protocol-final — the last observation defined by the measurement protocol;

response-final — the observable end of the platform-recorded response process.

 

The Summer 2026 data were sufficient to demonstrate that these concepts are not necessarily identical, but they were not sufficient to establish one common empirical response-final point.

 

For future research, this means that the core observation window and any extended follow-up window should be prospectively specified as separate components of the measurement design.

 

What did the feasibility audit show?

 

RQ4 was the primary pilot-study question.

The feasibility audit assessed six methodological domains. Its results were:

 

R01 conclusionData availability - SUPPORTED

Temporal feasibility - SUPPORTED WITH REDESIGN

Metric feasibility - SUPPORTED

Cross-wave comparability - RESTRICTED

Measurement validity - SUPPORTED WITH CONTROLS

Reproducibility and data governance - SUPPORTED

 

These categories are qualitative pilot-study decisions, not scores on a validated measurement scale. The overall conclusion is conditional rather than binary. Routine email-platform data can support exploratory longitudinal research, but research use requires additional controls that operational campaign reporting does not necessarily provide by default.

 

From operational reporting to research measurement

 

The main contribution of R01 is therefore methodological. The study does not introduce a new validated engagement metric and does not claim to demonstrate campaign effectiveness. Instead, it shows how standard platform metrics can be integrated into a more controlled longitudinal research architecture.

 

Four elements are especially important:

1. Temporal context
Every observation must be located relative to the sending unit’s own T0. Measurement order alone is insufficient.

2. Measurement validity
Platform-recorded events must remain conceptually distinct from higher-level human constructs such as reading, attention, learning or relationship quality.

3. Data provenance
Researchers need to know where an observation came from, under what platform conditions it was generated, and how it entered the analytical dataset.

4. Reproducibility
Source data, analytical layers, derived variables and reported results should form a traceable chain.

 

Within R01, this logic was implemented through an immutable master dataset, separated analytical layers and documented control fields for source, version, extraction context, bot-filter status and audit status where available.

 

What should change in the next study?

 

The pilot also identifies clear prospective design requirements.

 

A future ChildTech longitudinal communication study should use fixed post-send measurement windows relative to each sending unit’s T0, preserve Delivered_N and absolute unique-open and unique-click counts, document bot-filtering and reporting-interface conditions, and separate the standard observation protocol from extended follow-up.

​

Where technically, legally and ethically feasible, recipient-level event export could support more advanced event-time analyses. But platform activity should still remain only the first analytical layer.

 

Future research should distinguish among:

1. platform activity;

2. actual use and adaptation of professional content;

3. professional outcomes.

 

Questionnaires, interviews or other usage data would be required to investigate the second layer. Professional learning or practice outcomes would require separate operationalisation. They cannot be inferred from aggregate email analytics alone.

 

Why R01 matters

​

R01 demonstrates that the methodological challenge is not simply obtaining more data. The more fundamental issue is understanding what the data represent, when they were generated, how they were produced and what conclusions they can legitimately support. Routine platform analytics can become valuable research material when these conditions are explicitly controlled.

 

Without them, operational metrics remain useful communication indicators—but their research meaning is uncertain. With them, they can become part of an auditable longitudinal measurement system.

 

Reference

​

Piróth, I. (2026). Longitudinal Research Use of Routinely Generated Email-Platform Data in an ECEC Professional Context: An Exploratory Pilot Study. ChildTech Publications, R-series - Report 01 (R01). CHILDTECHLAB HUN - OKOSÓVODA R&D Subprogramme. BASIC EN v1.0 FINAL. DOI: to be inserted after assignment

 

The blog is available at: https://www.childtechlab.com/post/when-does-an-email-response-actually-happen-lessons-from-the-childtech-summer-2026-data 

​

For further regular professional updates, follow the ChildTechLab GLOBAL Facebook page