Deal Appetite Experiment

From a latent sales-ranking conjecture to stateful process prediction.

Companion repository: vbnovikov/deal-appetite-experiment

After looking at CRM data and business workflows across several clients, I started to wonder whether there was some underlying combination of factors that could reliably predict deal closing probability better than a simple lead score.

Things like number of touchpoints, response patterns, friction, time between interactions, stage movement, and other parts of the customer journey all seemed potentially useful when considered together.

That led to the idea that there may be a generalized ranking function over opportunities, with some part of the outcome determined by unobservable variation. If that were true, the useful question would be whether the recoverable part of that ranking could be estimated from observed CRM data, even when the full underlying process could not be observed directly.

Short Version

The original conjecture did not survive in its broadest form.

Synthetic data showed that a recoverable appetite ranking can be learned when the structure is explicitly built into the data-generating process. Real CRM opportunity fields, early interaction aggregates, and surface-level interaction text did not provide strong or stable out-of-sample ranking of terminal outcomes.

The more useful result appeared after changing the target from distant outcome prediction to process-local prediction.

On a real event log, current state alone predicted the next event with about $0.666$ accuracy. Adding process history increased accuracy to about $0.781$. Adding resource identity increased it to about $0.838$. Adding simple graph-derived relational context increased it again to about $0.846$.

So the final claim is narrower:

Business state may be better represented for decision-making as a history of connected entities and events than as the current values of those entities alone.

Hypothesis

$H_1$

There exists a recoverable ordering function over observable CRM variables:

such that $f(x_i)$ preserves useful relative ordering even when $\varepsilon_i$ is unobserved.

Here, $x_i$ means the feature vector for opportunity $i$: the list of measurements the model is allowed to see. Plain-language explanations of notation, feature vectors, feature groups, and preprocessing are included in Appendix A1, Appendix A2, Appendix A3, and Appendix A6.

$H_0$

Observable CRM variables do not recover a stable relative ordering of opportunities beyond noise and unobserved variation.

Experiment Setup

I generated 5,000 synthetic opportunities with four observable variables:

• fit
• urgency
• trust
• friction

Each variable was sampled independently from a standard normal distribution:

The observable component of appetite was defined as:

I then added an unobserved term:

giving total latent appetite:

$A_i$ was converted to a closing probability using the logistic function:

and the observed outcome was sampled as:

A logistic regression was then trained using only $f_i$, $u_i$, $t_i$, and $r_i$. The latent appetite, unobserved term, and true closing probability were withheld from the model.

The point of the synthetic setup is to separate what is observable from what is hidden. Appendix A2 explains how observable columns become model inputs, and Appendix B2 and Appendix B3 explain the ranking metrics used to evaluate the model.

Evaluation

The model was evaluated against three different targets.

The metrics used here are defined in plain language in Appendix B2 and Appendix B3.

1. Observed outcome

How well does the model rank deals that actually closed above deals that did not?

This is measured using ROC AUC against $Y_i$.

2. Total latent appetite

How closely does the model's ranking agree with the full latent quantity:

This includes the unobserved term, so perfect recovery should not be possible.

3. Systematic appetite

How closely does the model recover the ordering generated by the observable component:

This is the part of the synthetic process that should, in principle, be recoverable from the available data.

Results

The model achieved an ROC AUC of approximately:

against the observed close/no-close outcome.

That was encouraging, but the more important comparison was against the quantities that are known only because the data is synthetic.

The model had limited agreement with total latent appetite,

because $\varepsilon_i$ is deliberately hidden:

Its ranking of the systematic component $S_i$, however, was nearly perfect:

In the synthetic setting, the recoverable part of the ranking function was therefore recoverable from the observable variables.

The Problem

The synthetic experiment does not establish that a comparable latent variable exists in real CRM data.

The systematic component was defined directly from the same variables given to the model:

A model recovering that ordering therefore shows that the estimation procedure works under the assumptions of the simulation. It does not show that a comparable one-dimensional quantity exists in real CRM data.

The unobserved term creates another problem. In a real business process, $\varepsilon_i$ is not a clean random variable whose distribution is known. It can contain missing interactions, individual behavior, timing, relationships, information outside the CRM, and variables that were never recorded.

At that point, the conjecture loses a clear interpretation because the unobserved component can contain almost anything the model does not capture.

Real CRM Data

The next experiment used a real CRM sales dataset with resolved opportunities linked to account, product, and sales-team data.

The model only used information that would have been known while the opportunity was still open. Fields like the final close date or realized deal value were excluded because they would leak the outcome into the model and make the test meaningless.

To avoid leaking future information into the training set, the data was split chronologically:

The model was trained on earlier opportunities and evaluated on later ones.

This is a temporal holdout: the test cases occur later in time than the training cases. Appendix A7 explains why the same boundary also matters for preprocessing.

The baseline result was:

which is only slightly above random ranking.

Feature Ablation

I then removed and added groups of variables while keeping the model and temporal split fixed.

The results were:

Salesperson and team information added a small amount of predictive value. Adding account identity reduced performance on the future holdout set.

The static CRM fields available in this dataset therefore provided very little stable out-of-sample ranking power.

Interaction History

The weak performance of the static CRM model raised a more specific question: does the history of interactions with an opportunity contain useful information that is missing from the opportunity record itself?

The next experiment began with a comparison between static CRM data and static CRM data plus interaction-derived information. In practice, the available interaction records introduced a narrower test: whether coarse first-30-day activity summaries and surface-level interaction text contained stable ranking signal under temporal validation.

Static CRM Model

The first model estimates the probability that opportunity $i$ will be won using only static CRM features:

where $\mathbf{x}_i^{(s)}$ is the feature vector containing the static CRM information supplied to the model.

If this notation is unfamiliar, read $\mathbf{x}_i^{(s)}$ as "the static-data row for opportunity $i$ after it has been converted into numbers." Appendix A1 explains the notation, and Appendix A2 gives concrete examples.

The static feature set includes fields such as:

• product
• account
• sector
• salesperson
• regional office
• company revenue
• number of employees
• deal value
• engagement month

Categorical fields are one-hot encoded and numerical fields are standardized using statistics from the training set. The preprocessing details are included in Appendix A6 and Appendix A7.

Interaction-Derived Features

A natural next model would include information derived from the communication history associated with the opportunity:

The new term,

is another feature vector, this time derived from interactions rather than static CRM fields.

The superscripts here label feature groups. They are not exponents. $\mathbf{x}_i^{(s)}$ means static features, and $\mathbf{x}_i^{(b)}$ means behavioral or interaction-derived features.

Depending on the interaction record, these features can describe things such as:

• number of interactions
• time between interactions
• recency of communication
• direction of communication
• who initiated the interaction
• interaction type
• properties extracted from communication text

For example, an opportunity might have the following interaction history:

Day 1   Salesperson → Customer
Day 3   Customer → Salesperson
Day 8   Salesperson → Customer
Day 9   Customer → Salesperson

That sequence can be converted into numerical features such as:

interaction_count       = 4
customer_replies        = 2
salesperson_messages    = 2
days_since_last_contact = 1
mean_response_delay     = ...

Those values can then be encoded into $\mathbf{x}_i^{(b)}$ and supplied to the model alongside the static feature vector.

The combined expression can therefore be read as:

the probability that opportunity $i$ is won, given both its static CRM data and its interaction history.

The motivating comparison is whether the combined representation improves on static CRM data alone:

However, the available interaction records were not clean opportunity-level histories. Many interactions were better interpreted as relationship-level observations between salesperson and customer contact. The implemented baselines therefore tested a narrower question first: whether first-30-day activity summaries or first-30-day lexical content contained stable ranking signal under temporal validation.

Implementation details for preprocessing, modeling, and evaluation are included in the appendices, especially Appendix A6, Appendix A7, and Appendix B1.

Interaction History Results

The implemented interaction-history baselines tested:

for first-30-day activity summaries, and:

for first-30-day TF-IDF text features.

The initial interaction baselines did not find useful out-of-sample ranking signal. The activity-only temporal holdout produced:

and the text-only baseline produced:

Both are at or below the random-ranking benchmark:

This does not imply that interaction history is irrelevant. It shows that these initial representations did not recover stable outcome-ranking information in this dataset.

One likely issue is compression. Interaction history contains more detail than the static opportunity record, but much of that detail is lost before it reaches the model.

For example, a sequence such as:

Day 1   Salesperson → Customer
Day 2   Customer → Salesperson
Day 3   Customer → Salesperson
Day 12  Salesperson → Customer

might be reduced to features such as:

interaction_count       = 4
customer_messages       = 2
salesperson_messages    = 2
mean_response_delay     = ...
days_since_last_contact = ...

Those aggregate values preserve some information about the interaction history, but they no longer describe the sequence itself.

Two opportunities can therefore have similar aggregate features while having very different histories.

For example:

Opportunity A

Salesperson → Customer
Customer → Salesperson
Salesperson → Customer
Customer → Salesperson

and:

Opportunity B

Salesperson → Customer
Salesperson → Customer
Customer → Salesperson
Customer → Salesperson

can produce similar counts even though the order of events is different.

The same issue applies to timing. Four interactions spread evenly across a week describe a different process from four interactions followed by a month of silence, even if both opportunities have the same total interaction count.

This led to the next question: Is the useful information contained in the trajectory of an opportunity rather than in aggregate features calculated from that trajectory?

Sequential Customer Journeys

The interaction-history experiment reduced a sequence of events to summary variables such as interaction count, recency, and response timing. That loses information about the order in which those events occurred.

The next experiment tested whether preserving more of the sequence improves prediction.

Instead of treating an opportunity as one static feature vector, each customer journey is represented as an ordered series of events.

This is the sequence version of the same idea. A feature vector stores a fixed list of inputs for one example; a sequence stores an ordered list of events for one example. Appendix A2 gives the feature-vector version first, and Appendix A5 shows how histories are converted into model inputs.

For session $i$, let:

where:

• $E_i$ is the complete observed journey for session $i$
• $e_{i1}$ is the first event
• $e_{i2}$ is the second event
• $n_i$ is the total number of events observed in that session

A simple journey might look like:

home
→ product_page
→ cart
→ checkout
→ confirmation

The order is important because a customer who reaches checkout has followed a different process from someone whose session ends on the product page, even if both generated the same number of events.

Journey Prefixes

The model should only use information that would have been available at the time a prediction was made.

For that reason, the experiment does not give the model the complete journey at once. It constructs progressively longer prefixes of the journey.

The first $k$ events are represented as:

For example:

k = 1

home

k = 2

home
→ product_page

k = 3

home
→ product_page
→ cart

k = 4

home
→ product_page
→ cart
→ checkout

At each point, the model estimates:

Here:

• $Y_i = 1$ means that session $i$ eventually results in a purchase
• $E_i^{(k)}$ is everything observed in that session up to event $k$
• $\hat{p}_i^{(k)}$ is the estimated probability of eventual conversion based only on that partial journey

The expression can be read as:

the estimated probability that session $i$ eventually converts, given the first $k$ events observed in its customer journey.

The experiment then compares:

If the ordering becomes more accurate as $k$ increases, additional process history is providing useful information about the eventual outcome.

Avoiding Outcome Leakage

The dataset contains a Purchased field on every row in a session.

That creates an important problem:

If a session eventually results in a purchase, earlier rows in the same session can also contain:

Purchased = 1

even though the purchase had not happened yet.

Using that field as an input would effectively tell the model the answer in advance.

The confirmation event creates a similar issue. Reaching confirmation means the purchase has already occurred, so including it in the predictive history would make the experiment trivial.

For that reason:

• Purchased is used only as the final target
• confirmation is treated as a post-outcome event
• only events that occur before confirmation are used as model inputs

This keeps the question meaningful: given only what had happened so far, could the model rank sessions by their eventual probability of conversion?

This is the same leakage rule used throughout the experiments: do not let information from after the prediction time enter the inputs. Appendix A7 describes the rule in preprocessing terms.

Constructing the Sequences

The event data is first ordered by session and timestamp:

customer_journey["Timestamp"] = pd.to_datetime(
    customer_journey["Timestamp"]
)

customer_journey = (
    customer_journey
    .sort_values(["SessionID", "Timestamp"])
    .reset_index(drop=True)
)

Each event is then numbered within its session:

customer_journey["event_number"] = (
    customer_journey
    .groupby("SessionID")
    .cumcount()
    + 1
)

A session such as:

Session 42

12:01   home
12:03   product_page
12:07   cart
12:10   checkout

therefore becomes:

event_number   page
1              home
2              product_page
3              cart
4              checkout

This makes it possible to construct the first event, first two events, first three events, and so on without using anything that occurred later in the session.

The important change in this experiment is that the prediction is now conditioned on an ordered process history rather than only on a static record or a set of aggregate interaction features.

Sequential Journey Results

Ranking performance did not improve meaningfully as more of the customer journey became visible.

The pre-confirmation prefix models produced:

All four results are below the random-ranking benchmark:

So this experiment did not support the idea that these customer-journey prefixes recover a stable ranking of eventual conversion.

It did, however, expose an important interpretation problem for any future sequence experiment.

Consider these two partial journeys:

Session A

home
→ product_page
→ cart

Session B

home
→ product_page
→ cart

Both sessions have the same current state: cart.

If a model distinguishes sessions mainly because one has reached cart while another remains on home, then it has learned funnel position. That does not show that the path taken to reach the current state contains additional information.

The distinction can be written as:

versus:

where:

• $S_i$ is the current state of session $i$
• $H_i$ is the history that led to that state
• $Y_i = 1$ means the session eventually converts

The first expression asks:

What is the probability of conversion given where the session is now?

The second asks:

What is the probability of conversion given both where the session is now and how it got there?

That is a much stronger test.

Current State as a Baseline

Suppose one customer has reached checkout after moving directly through the funnel:

home
→ product_page
→ cart
→ checkout

Another customer is also currently at checkout, but arrived there after repeated navigation between earlier states:

home
→ product_page
→ home
→ product_page
→ cart
→ product_page
→ cart
→ checkout

A conventional CRM-style snapshot could represent both sessions as:

current_state = checkout

The current-state model therefore receives the same main piece of information for both.

A history-aware representation can preserve the different trajectories.

The relevant comparison becomes:

If adding $H_i$ improves prediction while $S_i$ is held constant, then the trajectory itself contains information that is lost in the current-state representation.

This became the basis for the next experiments.

Put simply, the current state is like a summary column, while the history is the set of steps that produced that summary. The next experiments ask whether that summary column has thrown away useful information.

State Transitions

To isolate the effect of history, the next experiment uses an explicit state-transition process.

Instead of predicting only whether a deal eventually closes, the process is represented as a sequence of states:

Here, $S_{i,t}$ is the state of process $i$ at time $t$.

A simplified sales process might look like:

New
→ Contacted
→ Qualified
→ Proposal
→ Won

but another opportunity might follow:

New
→ Contacted
→ Qualified
→ Contacted
→ Qualified
→ Proposal
→ Lost

At some point, both opportunities can occupy the same state even though their histories are different.

That gives us a cleaner question:

Among opportunities that are currently in the same state, does their previous transition history help predict what happens next?

Formally, the baseline uses only the current state:

The history-aware model uses the current state and the preceding trajectory:

The first expression asks for the probability of the next state given only the current state.

The second asks whether knowing the path taken to reach that state changes the probability of what happens next.

This removes much of the ambiguity from the customer-journey experiment. The comparison is no longer between customers at obviously different points in a funnel. It compares cases that occupy the same current state and asks whether their histories still matter.

That is why the later experiments are stricter than ordinary funnel-stage prediction. They compare like with like: same current state, different prior path.

Within-State Transition History

The previous experiment showed that later process states are easier to rank, but that does not tell us whether the history leading to a state matters.

To test that directly, I built a synthetic state-transition process where opportunities move through a small sales funnel:

Each opportunity starts in new and moves through the process one step at a time.

At any non-terminal state, it can:

• advance
• remain in the same state
• regress
• become lost

For example:

new
→ engaged
→ qualified
→ proposal
→ won

is one possible trajectory, while:

new
→ engaged
→ qualified
→ engaged
→ qualified
→ proposal
→ lost

is another.

A Hidden Propensity

Each synthetic opportunity is assigned an unobserved value:

which affects how likely it is to move forward, stall, regress, or become lost.

Higher values of $A_i$ make forward movement more likely and loss less likely.

The transition process can be written as:

where:

• $S_{i,t}$ is the current state of opportunity $i$
• $S_{i,t+1}$ is its next state
• $A_i$ is the hidden propensity assigned to that opportunity

The expression can be read as:

the probability of the opportunity's next state, given its current state and its unobserved propensity.

The model never receives $A_i$. It only observes what happened to the opportunity.

Generating Transition Probabilities

Each possible transition is first assigned a score.

For example:

The coefficients control how strongly the hidden propensity affects each type of transition.

These scores are not probabilities yet. They can take any real value.

To convert them into probabilities that add up to $1$, the experiment uses the softmax function:

Here, $\mathcal{T}(S_{i,t})$ is the set of transitions that are available from the current state.

For example, an opportunity in qualified might be able to:

advance → proposal
stay    → qualified
regress → engaged
exit    → lost

Softmax converts the four transition scores into four probabilities whose total is:

One of those transitions is then sampled, producing the next observed state.

This process repeats until the opportunity reaches either won or lost.

Why Use Synthetic Data Again?

The question is no longer whether a general latent deal score exists.

The synthetic process gives us a controlled environment where we know that past transitions contain information about an unobserved variable.

That lets us test a specific representation question:

If two opportunities are currently in the same state, can their previous transitions help distinguish them?

Suppose two opportunities are both currently qualified.

The first followed:

new
→ engaged
→ qualified

The second followed:

new
→ engaged
→ qualified
→ engaged
→ qualified

A current-state representation gives both of them:

current_state = qualified

A history-aware representation can distinguish the two trajectories.

The comparison is therefore:

against:

where $H_{i,t}$ contains the transitions observed before time $t$.

If the history-aware model performs better among opportunities occupying the same current state, then current state is losing information about the process that produced it.

That is the specific property being tested here.

Within-State Ranking Results

The history-aware model was evaluated separately within the engaged, qualified, and proposal states.

The results were:

All three are above the random-ranking benchmark:

Because each model is evaluated only among opportunities that occupy the same current state, differences in pipeline stage cannot explain the ranking.

For example, every opportunity in the proposal evaluation set is already at proposal. The model therefore cannot perform well simply by learning that proposal is generally closer to won than engaged.

The information available to the model comes from the history that produced that state, including features such as:

• number of forward transitions
• number of regressions
• number of repeated states
• total number of transitions
• net progress
• regression rate
• most recent transition

So, within the synthetic process, two opportunities can both be at:

proposal

while their histories imply different probabilities of eventually reaching won.

For example:

Opportunity A

new
→ engaged
→ qualified
→ proposal

and:

Opportunity B

new
→ engaged
→ qualified
→ engaged
→ qualified
→ proposal

have the same current state but different trajectories.

The result shows that the trajectory contains information that is lost when both opportunities are represented only as:

current_state = proposal

Limitation

This result still comes from a synthetic process.

The hidden propensity $A_i$ was deliberately constructed to affect transition probabilities. An opportunity with a higher $A_i$ is more likely to advance and less likely to regress or become lost.

Transition history is therefore expected to contain information about $A_i$.

The result does not establish that real sales or business processes behave this way. It shows that if an unobserved factor affects how entities move through a process, their observed transition history can preserve information about that factor even after their current state is known.

That gives us a real-data question:

among cases observed at the same point in a real business process, does the history leading to that point contain useful information about what eventually happens?

Testing Transition History on Real Process Data

The synthetic state-transition experiment shows that history can matter when the process is explicitly constructed so that previous transitions contain information about an unobserved variable.

That still does not tell us whether the same thing happens in real business processes.

The next experiment uses the BPI Challenge 2017 event log, which contains event-level histories for real loan applications. Unlike a normal CRM snapshot, the dataset records the sequence of activities associated with each application over time.

A simplified application history might look like:

A_Create Application
→ A_Submitted
→ A_Concept
→ A_Accepted
→ O_Create Offer
→ O_Sent
→ A_Pending

Another application may reach some of the same intermediate states through a different sequence.

That makes it possible to repeat the within-state test using real process history.

Defining the Outcome

Each application eventually reaches one of three terminal states:

A_Pending
A_Denied
A_Cancelled

For the binary experiment, successful completion is defined as:

The target is determined only from the final terminal event.

Intermediate activities such as A_Accepted are not treated as successful outcomes because the application can still fail later in the process.

This distinction between intermediate events and final labels is another form of leakage control. The model is not allowed to treat a promising middle state as if it were already the final answer.

Comparing Applications at the Same State

The key requirement is that the applications being compared must occupy the same observable process state.

Suppose two applications have both reached:

O_Sent

At that point, their current state is identical.

For a checkpoint state $s$, define:

$\mathcal{C}_s$ is simply the set of applications that reached the same checkpoint.

For every application in that set, the model is allowed to use only the events that occurred before or at the first occurrence of that checkpoint.

Anything that happens later is hidden.

This gives each application a history:

while holding the final observed state fixed:

The question is then:

Can the history $H_{i,t}$ help rank applications that are all currently at the same state $s$?

If it can, then the current state alone is not a sufficient description of the process.

Why the First Occurrence Matters

Some applications can return to the same activity more than once.

For example:

A_Submitted
→ A_Concept
→ A_Accepted
→ A_Concept
→ A_Accepted

If we simply chose an arbitrary occurrence of A_Accepted, some applications would have much more future information available than others.

The experiment therefore uses the first occurrence of the chosen checkpoint.

Everything after that point is excluded.

This gives every application the same prediction condition:

what could have been known the first time this application reached this state?

Temporal Validation

The data is again split chronologically.

Applications that started earlier are used for training, while later applications are reserved for testing.

This matters because a random split could allow the model to learn process patterns from future cases and use them to predict earlier ones.

The real question is whether historical process data can help rank applications that occur later in time.

Within-State AUC

For each checkpoint state $s$, ranking performance is measured separately:

Because every application in that evaluation set already occupies the same current state, the state itself cannot explain differences in ranking.

A value of:

would mean the history provides no useful ordering within that state.

A value above:

would indicate that the path leading to the checkpoint contains information about the eventual terminal outcome.

That is a much stronger test than asking whether applications further along in the process are more likely to succeed.

Real Within-State Results

The first history-only models used aggregate process features such as event counts, elapsed time, repeated activities, offer activity, and workflow activity.

The A_Submitted checkpoint was excluded before modeling because almost every application reached it through the same observed history. There was essentially no variation to rank on.

The remaining checkpoints were:

• A_Validating
• A_Incomplete

The history-only models produced:

and:

Both are above the random-ranking benchmark:

Because each model is evaluated only among applications at the same checkpoint, the result cannot be explained by one application simply being further through the process than another.

The difference comes from what happened before the checkpoint.

The effect is still fairly weak. An AUC around $0.58$ does not support accurate within-state ranking, but it does suggest that some information about the eventual outcome survives in the prior process history.

Preserving More of the Sequence

The first real-data model still compresses history into aggregate counts.

For example, these histories:

A → B → C → B → C

and:

A → B → B → C → C

can produce similar event counts despite having different transition structures.

The next version therefore added features describing adjacent transitions:

Instead of recording only how often an activity occurred, the model could also record transitions such as:

A_Validating → A_Incomplete
A_Incomplete → W_Call incomplete files
O_Create Offer → O_Sent

The sequence-aware model produced:

and:

Compared with the aggregate-history model:

for A_Validating, while:

for A_Incomplete.

Preserving adjacent transition structure therefore improved one checkpoint slightly and made the other worse.

There is some recoverable information in the process history, but simply adding more sequence detail does not produce a consistently stronger ranking model.

What This Actually Supports

The original conjecture was much broader: that opportunity outcomes might be recoverable through a generalized ranking function representing some latent transaction propensity.

These experiments do not provide strong evidence for that claim.

The real-data results are much narrower. Applications observed at the same process state can still differ in ways that are partially recoverable from the history that brought them there.

A useful way to state that is:

where the current state $S_{i,t}$ is produced by some preceding history $H_{i,t}$.

Representing only $S_{i,t}$ therefore compresses that history into a much smaller description of the process.

Different histories can map to the same current state:

while:

Some information that distinguishes $H_a$ from $H_b$ is necessarily absent from the current-state representation.

The empirical question is whether any of that discarded information matters for the decision being made.

In the BPI experiment, at least some of it does.

The effect is modest, but the result is enough to motivate a different prediction target: rather than trying to predict a distant terminal outcome such as won, lost, or successful completion, can the current process history predict what happens next?

Next-State Prediction

The terminal-outcome experiments produced only modest within-state ranking performance. The next experiment changes the target.

Instead of asking whether an application will eventually succeed, the model predicts the next observable process event.

For case $i$ at time $t$, let:

represent the current state,

the process history observed so far, and:

the relational context surrounding the case.

The target is:

so the full prediction problem becomes:

This can be read as:

the probability of the next process state, given the current state, the history so far, and the surrounding relational context.

The notation is compact, but the idea is ordinary supervised learning: each row contains what was known at time $t$, and the label is what happened at time $t+1$. Appendix A5 translates this into a simple table-style example.

Constructing Prediction Snapshots

The BPI event log is first ordered by application and timestamp:

bpi = (
    bpi
    .sort_values(
        ["case:concept:name", "time:timestamp"]
    )
    .reset_index(drop=True)
)

Each event is then assigned its position within the application:

bpi["event_position"] = (
    bpi
    .groupby("case:concept:name")
    .cumcount()
)

The next event is created by shifting the activity column one row backward within each application:

bpi["next_activity"] = (
    bpi
    .groupby("case:concept:name")["concept:name"]
    .shift(-1)
)

This turns an event sequence such as:

A_Submitted
→ A_Concept
→ A_Accepted
→ O_Create Offer

into supervised examples:

current: A_Submitted
target:  A_Concept

current: A_Concept
target:  A_Accepted

current: A_Accepted
target:  O_Create Offer

The last event in each application is removed because there is no following event to predict:

event_snapshots = bpi[
    bpi["next_activity"].notna()
].copy()

Temporal Train/Test Split

The split is performed at the application level using case start time.

First, the start time of each application is calculated:

case_start_times = (
    bpi
    .groupby("case:concept:name")["time:timestamp"]
    .min()
)

A chronological cutoff is then chosen:

cutoff = case_start_times.quantile(0.75)

Earlier applications form the training set:

train_snapshots = event_snapshots[
    event_snapshots["case_start_time"] < cutoff
].copy()

Later applications form the test set:

test_snapshots = event_snapshots[
    event_snapshots["case_start_time"] >= cutoff
].copy()

This preserves the same basic rule used throughout the experiments: the model should learn from the past and be evaluated on later cases.

Appendix A7 gives the general version of this rule, and Appendix B1, Appendix B4, and Appendix B6 explain why the reported next-event metrics are accuracy, macro F1, and weighted F1 rather than only AUC.

State-Only Baseline

The first baseline uses only the current activity.

For every state $s$, the training data is used to find the next event that followed that state most often:

The $\arg\max$ operator means:

choose the value of $j$ that gives the largest probability.

In code:

state_next_mode = (
    train_snapshots
    .groupby("concept:name")["next_activity"]
    .agg(
        lambda x: x.value_counts().idxmax()
    )
)

If A_Concept was most often followed by A_Accepted in the historical training data, then the baseline predicts:

A_Concept → A_Accepted

whenever it encounters A_Concept.

Those predictions are applied to the test set:

test_snapshots["state_only_prediction"] = (
    test_snapshots["concept:name"]
    .map(state_next_mode)
)

The resulting temporal accuracy was:

with complete coverage of the test set.

So current state alone correctly predicts the next event about two-thirds of the time.

That is already much stronger than the earlier attempts to predict a distant terminal outcome.

Adding Process History

The next model includes compact information about the path observed so far.

The history representation includes:

• event position
• elapsed time since application creation
• previous activity
• number of unique activities observed
• number of repeated activities
• cumulative application-event count
• cumulative offer-event count
• cumulative workflow-event count

The prediction changes from:

to:

A simplified feature set might look like:

current_activity        = A_Validating
previous_activity       = A_Incomplete
event_position          = 12
elapsed_hours           = 47.3
unique_activity_count   = 8
repeated_activity_count = 4
offer_event_count       = 2
workflow_event_count    = 5

Categorical features such as current_activity and previous_activity are one-hot encoded. Numerical features are standardized. Appendix A6 shows why both transformations are needed before a standard classifier can read the data.

The model is built with scikit-learn:

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.linear_model import SGDClassifier

The preprocessing step can be defined as:

preprocessor = ColumnTransformer(
    transformers=[
        (
            "categorical",
            OneHotEncoder(handle_unknown="ignore"),
            categorical_features,
        ),
        (
            "numeric",
            StandardScaler(),
            numeric_features,
        ),
    ]
)

The classifier and preprocessing are then combined into a single pipeline:

model = Pipeline(
    steps=[
        ("preprocessor", preprocessor),
        (
            "classifier",
            SGDClassifier(
                loss="log_loss",
                max_iter=2000,
                random_state=42,
            ),
        ),
    ]
)

The important point is that preprocessing is learned only from the training data.

The model is then fit with:

model.fit(
    X_train,
    y_train
)

and next-event predictions are generated for the temporal holdout:

history_prediction = model.predict(
    X_test
)

The resulting accuracy was:

Compared with the state-only baseline:

which is an absolute improvement of:

or about 11.5 percentage points.

The history of the process is therefore much more useful for predicting the next event than it was for predicting a distant final outcome.

Relational Context Ablation

The next step was to test which additional forms of context actually mattered.

Rather than adding every available field at once, the experiment uses ablation.

Ablation means training several otherwise similar models while adding or removing specific groups of features. If performance changes materially when one group is introduced, that group is contributing useful predictive information.

The main representations were:

State + History
State + History + Case
State + History + Resource
State + History + Case + Resource

The models were evaluated with:

• accuracy
• macro F1
• weighted F1

These are classification metrics rather than ranking metrics because the target is now a specific next event, not a binary win/loss outcome. Appendix B1 explains the difference.

The results were:

Adding case-level information produced only a small improvement.

Adding resource identity produced a much larger one:

What Does "Resource" Mean?

In the BPI process log, events can be associated with the resource responsible for that activity.

That might represent a user, employee, team member, or operational handler.

So a row can contain information conceptually like:

case      = Application_4821
activity  = A_Validating
resource  = User_37

When resource identity is added to the model, it can learn that different resources are associated with different transition patterns.

That does not establish that the resource causes the transition.

Resource identity can also capture:

• specialization
• role
• assignment rules
• case complexity
• organizational structure
• operational routing
• workload patterns

The ablation establishes only that this information helps predict what happens next. Metric definitions for accuracy, macro F1, and weighted F1 are included in Appendix B4, Appendix B5, and Appendix B6.

Graph-Derived Context

Resource identity is relational information, but encoding a resource as a category still treats it as a label.

For example:

resource = User_37

does not tell the model how User_37 is connected to other applications or resources.

The event log can also be represented as a graph.

A simplified local structure might look like:

Application A
├── handled by → Resource 12
├── handled by → Resource 41
└── contains   → Event ...

Application B
├── handled by → Resource 12
└── contains   → Event ...

Because Resource 12 appears in both applications, the two cases are connected through a shared node.

Graph-derived features summarize parts of this local structure and add them to the existing state, history, and resource representation.

At this point, the feature vector is no longer just a set of fields from one row. It also includes simple summaries of the case's local graph context.

The final comparison was:

The graph-derived features produced a smaller additional improvement:

in accuracy, while macro F1 increased:

The full progression was:

The largest improvement came from process history.

Resource identity added another substantial increase.

Graph-derived relational context added a smaller but consistent improvement across the reported metrics.

What This Suggests for CRM Architecture

The original appetite conjecture did not survive the experiments in a useful general form.

Static CRM data performed poorly on future opportunity ranking. Adding interaction summaries helped only modestly. Sequential history became more useful once the target moved closer to the process itself, and the strongest result came from predicting the next transition using current state, process history, resource identity, and local relational context.

The progression in the final experiment was:

for:

current state
→ state + history
→ state + history + resource
→ state + history + resource + graph context

This does not show that every CRM should use a graph database.

It does suggest that the representation used for prediction matters.

The Relational Representation

A conventional relational CRM might store an opportunity across tables such as:

opportunities
-------------
id
account_id
owner_id
stage
value
created_at
updated_at

with related information in other tables:

contacts
activities
users
accounts
documents
stage_history

There is nothing inherently incapable about this representation. SQL can reconstruct relationships and history through joins.

For example:

SELECT
    o.id,
    o.stage,
    a.type,
    u.id AS owner_id,
    h.previous_stage,
    h.changed_at
FROM opportunities o
LEFT JOIN accounts a
    ON a.id = o.account_id
LEFT JOIN users u
    ON u.id = o.owner_id
LEFT JOIN stage_history h
    ON h.opportunity_id = o.id;

The issue is not whether a relational database can store the information. It can.

The question is what representation should be exposed to a decision model when the decision depends on a connected process unfolding over time.

A single opportunity row might say:

opportunity_id = 1842
stage          = proposal
owner          = user_17
value          = 120000

but the process that produced that row may contain:

created
→ assigned to user_04
→ contacted
→ qualified
→ assigned to user_17
→ proposal created
→ customer replied
→ proposal revised
→ proposal

The current row is one projection of that history.

A Graph Representation

The same process can be represented in terms of entities and their relationships:

Opportunity 1842
├── BELONGS_TO → Account 93
├── OWNED_BY → User 17
├── INVOLVES → Contact 51
├── HAS_EVENT → Call 882
├── HAS_EVENT → Email 901
├── HAS_EVENT → Proposal 44
└── HAS_STATE → Proposal

Events can also connect to each other through time:

Call 882
→ FOLLOWED_BY → Email 901
→ FOLLOWED_BY → Proposal 44
→ FOLLOWED_BY → Revision 46

and resources can connect multiple cases:

User 17
├── HANDLED → Opportunity 1842
├── HANDLED → Opportunity 2201
└── HANDLED → Opportunity 2390

That makes relationships part of the representation itself rather than something reconstructed only when a particular query needs them.

For a prediction at time $t$, the available context can be thought of as a local subgraph:

where:

• $V_{i,t}$ is the set of relevant entities known at time $t$
• $E_{i,t}$ is the set of relationships between those entities

A model can then condition on:

The experiments do not establish that this is always the best representation. They show that information represented by $H_{i,t}$ and parts of $G_{i,t}$ improved next-state prediction on the process data tested here.

Appendix A2 and Appendix A6 give the preprocessing view of this: graph context still has to be converted into model-readable features before a standard classifier can use it.

Current State Is a Compression

The clearest result from the experiments is that current state can discard information.

Let the complete process history before time $t$ be:

and let the CRM derive a current state from that history:

Many different histories can produce the same state.

For two cases $a$ and $b$:

while:

A simple example is:

Case A

new
→ engaged
→ qualified
→ proposal

and:

Case B

new
→ engaged
→ qualified
→ engaged
→ qualified
→ proposal

Both can be stored as:

stage = proposal

but the histories are different.

Whether that difference matters depends on the task.

For terminal-outcome ranking in the real data, the additional information was weak.

For next-state prediction, it mattered considerably more.

That distinction is important. There is no reason to assume that richer process state will improve every prediction problem equally.

A Stateful CRM

The architecture I find more interesting after these experiments is a CRM where the current record remains useful, but it is treated as one view over an underlying process history.

Conceptually:

entities
    +
events
    +
relationships
    +
time
    ↓
current process state
    ↓
prediction / decision

An event might look like:

{
  "type": "proposal_sent",
  "opportunity_id": "1842",
  "actor_id": "user_17",
  "contact_id": "contact_51",
  "timestamp": "2026-08-31T15:42:00Z"
}

Another event could record a state transition:

{
  "type": "stage_changed",
  "opportunity_id": "1842",
  "from": "qualified",
  "to": "proposal",
  "actor_id": "user_17",
  "timestamp": "2026-08-31T15:44:00Z"
}

The current opportunity record can still be derived and queried normally:

stage = proposal
owner = user_17

but the underlying history remains available when a decision depends on how that state was reached.

This also avoids requiring every downstream model to treat the CRM row as a complete description of the business process.

What Survived

I started these experiments looking for a generalized ranking function for deal appetite.

The synthetic version was recoverable because the latent structure had been placed into the simulation. Real CRM data did not provide evidence for a comparably useful scalar, and increasingly detailed histories did not produce a strong general model of terminal outcomes.

The more consistent result concerned process state.

Static CRM fields were weak for future deal ranking. Real transition history retained modest information within the same process state. When the target was changed to the next observable transition, history produced a much larger improvement, resource identity added more, and graph-derived relational context added a smaller additional gain.

The experiments therefore give me a narrower hypothesis to work with:

Business state may be better represented for decision-making as a history of connected entities and events than as the current values of those entities alone.

That is a claim about representation, not about Neo4j specifically.

A relational database can store the same underlying information. A graph may simply provide a more direct model when the questions being asked depend on paths, shared entities, transitions, and local context.

The remaining question is whether that advantage survives outside this dataset and whether it is large enough to justify the additional complexity of a graph-native representation.

Hypothesis Revisited

The original hypotheses were:

$H_1$

There exists a recoverable ordering function over observable CRM variables:

such that $f(x_i)$ preserves useful relative ordering even when $\varepsilon_i$ is unobserved.

$H_0$

Observable CRM variables do not recover a stable relative ordering of opportunities beyond noise and unobserved variation.

The experiments do not provide enough evidence to support $H_1$ in this general form.

The synthetic experiment recovered the ranking structure that had been explicitly built into the data-generating process. That established that the estimation procedure could recover the systematic component under the assumptions of the simulation, but it did not establish that a comparable one-dimensional ordering function exists in real CRM data.

On real CRM data, static opportunity features produced weak out-of-sample ranking of terminal outcomes. Interaction history and process-history features improved performance in some experiments, but the gains were not strong or consistent enough to establish a stable generalized ordering of opportunities.

I therefore would not reject $H_0$ for the original conjecture.

The later experiments support a narrower result.

Process history can retain information that is absent from the current-state representation, and that information becomes substantially more useful when the target is the next observable process transition rather than a distant terminal outcome.

In the final experiment, next-event accuracy increased from:

using current state alone, to:

after adding process history, then to:

after adding resource identity, and finally to:

after adding graph-derived relational context.

Those results do not rescue the original $H_1$. They point to a different hypothesis about how business processes should be represented for prediction and decision-making.

Conclusion

The deal-appetite conjecture began as a search for a generalized opportunity-ranking function. The synthetic experiment showed that such a function can be recovered when the recoverable structure is deliberately built into the data-generating process. The real-data experiments did not show that an analogous scalar exists in ordinary CRM records.

The more durable finding is about representation.

When the target is a distant terminal outcome, richer history is only weakly useful in the datasets tested here. When the target is the next observable process transition, history and relational context matter much more.

That suggests a narrower and more useful direction:

Business prediction should often be framed around the stateful process that produces the current CRM record, not only around the current record itself.

A graph database is not required for that claim to be true. But graph-shaped representations make the relevant objects easier to name directly: events, actors, accounts, contacts, offers, states, transitions, and the paths between them.

The next question is whether richer graph-native representations can improve decision support in real CRM systems enough to justify their added complexity.

Appendices

These appendices are numbered by topic. Appendix A covers notation, feature construction, and preprocessing. Appendix B covers evaluation metrics. Appendix C covers data and reproducibility.

Appendix A1: Libraries and Notation

This appendix explains the notation and library mechanics used throughout the experiments. It is meant to make the article readable even if the reader has only seen introductory statistics or discrete math.

The experiments were written in Python and run in Jupyter.

The main libraries were:

• pandas for loading, joining, filtering, grouping, and transforming CRM and event-log data
• NumPy for random-number generation and vectorized numerical operations in the synthetic experiments
• scikit-learn for preprocessing, model fitting, and evaluation
• SciPy for Spearman rank correlation
• Matplotlib for plots and visual inspection of results

Notation guide

The notation is compact because the same structure appears many times.

A subscript tells us which example we are talking about.

For example:

means "the feature vector for example 7."

A superscript in this article is usually a label, not a power.

For example:

means the static CRM feature vector for opportunity $i$.

It does not mean $\mathbf{x}_i$ multiplied by itself.

The hat symbol means estimated by the model.

For example:

means the true probability in the synthetic experiment, while:

means the model's estimated probability.

The vertical bar means "given" or "conditional on."

For example:

means:

the probability that example $i$ has outcome $1$, given the features in $\mathbf{x}_i$.

The membership symbol means "is an element of."

For example:

means:

$\sigma$ is chosen from the interval between $0$ and $5$.

In the noise-sensitivity experiment, this was implemented as a finite grid:

noise_levels = np.arange(0, 5.25, 0.25)

So the computer did not test every real number between $0$ and $5$. It tested:

0.00, 0.25, 0.50, ..., 5.00

The normal-distribution notation:

means:

draw $x_i$ from a normal distribution with mean $0$ and standard deviation $1$.

Appendix A2: Feature Vectors

A model cannot read a CRM row the way a person reads a spreadsheet. It needs a list of values.

For one opportunity $i$, that list is called a feature vector:

The bold $\mathbf{x}$ means that the input contains several values, not just one value.

Suppose one opportunity has:

product          = GTX Pro
sector           = Technology
sales_price      = 82,000
employees        = 220
is_subsidiary    = 0
engage_month     = 3

Before a standard model can use that row, text values must be converted into numbers. After preprocessing, the model might receive this vector:

feature name          value
-------------------   -----
product_GTX_Basic       0
product_GTX_Pro         1
product_MG_Special      0
sector_Finance          0
sector_Technology       1
scaled_sales_price      0.42
scaled_employees       -0.18
is_subsidiary           0
engage_month            3

The same thing can be written as a mathematical vector:

Each row of the vector is one input feature.

When there are many opportunities, the model receives a table of feature vectors:

Each row is one example. Each column is one feature.

A small feature table might look like this:

The model is trained on the feature columns:

and the label column:

For a binary sales model:

y_i = 1  if the deal was won
y_i = 0  if the deal was lost

So the model learns a relationship of the form:

or, for probability prediction:

where $\hat{p}_i$ is the model's estimated probability.

Appendix A3: Feature Groups

The article uses superscripts to label different groups of inputs.

For example:

means static CRM features, such as product, sector, salesperson, and deal value.

means behavioral or interaction-derived features, such as interaction count, reply timing, or channel mix.

The combined model can therefore be read as:

which means:

the probability that opportunity $i$ is won, given its static CRM features and behavioral features.

Appendix A4: Logistic Regression

In a binary model, logistic regression first converts the feature vector into a single score:

Here:

• $x_{i1}$ is the first feature for example $i$
• $x_{i2}$ is the second feature for example $i$
• $k$ is the number of features
• $\beta_0$ is the intercept
• $\beta_1,\ldots,\beta_k$ are learned coefficients

A tiny worked example:

features:
    product_GTX_Pro      = 1
    sector_Technology    = 1
    scaled_sales_price   = 0.42

learned coefficients:
    intercept            = -0.30
    product_GTX_Pro      = 0.50
    sector_Technology    = 0.20
    scaled_sales_price   = 0.80

The model score is:

That score is not yet a probability. Logistic regression applies the logistic function:

So:

The model would estimate about a $67.6\%$ probability of the positive class.

In code, scikit-learn does this after the model has been fit:

predicted_probability = model.predict_proba(X_test)[:, 1]

The [:, 1] means:

take every row, and take column 1 of the probability output.

For a binary classifier, column 0 is usually the probability of class 0, and column 1 is the probability of class 1.

Appendix A5: Histories as Feature Vectors

For process-history models, the raw object is no longer just one CRM row. It is a history:

A_Submitted
-> A_Concept
-> A_Accepted
-> O_Create Offer

To use a standard classifier, that history still has to be converted into features.

For example:

current_activity        = A_Accepted
previous_activity       = A_Concept
event_position          = 3
elapsed_hours           = 12.7
unique_activity_count   = 3
offer_event_count       = 0
workflow_event_count    = 1

Those values form the feature vector for that prediction time.

For next-event prediction, each training row contains information available at time $t$, and the label is the event that happened at time $t+1$.

This is the same supervised-learning structure as before:

The important rule is that features must be computed only from information available at or before time $t$.

Appendix A6: Categorical and Numerical Preprocessing

For static CRM models, categorical and numerical fields require different preprocessing.

Categorical fields are fields with names or categories, such as:

product = GTX Pro
sector  = Technology
office  = West

They were converted into numeric columns using OneHotEncoder:

from sklearn.preprocessing import OneHotEncoder

For example:

product = GTX Pro

may become:

GTX Basic   GTX Pro   MG Special
    0          1           0

The setting:

OneHotEncoder(handle_unknown="ignore")

means:

if a category appears in the test set that was not seen during training, do not crash; encode it as all zeros for that feature group.

Numerical fields are already numbers, such as revenue, employee count, elapsed time, or event count.

They were standardized using StandardScaler:

from sklearn.preprocessing import StandardScaler

Standardization converts a raw value $x$ into:

where $\mu$ is the mean of that feature in the training data and $\sigma$ is its standard deviation.

For example, if the average deal value in the training set is $50{,}000$ and the standard deviation is $20{,}000$, then a deal worth $70{,}000$ becomes:

That means the deal is one training-set standard deviation above the training-set average.

A deal worth $30{,}000$ would become:

That means the deal is one training-set standard deviation below the training-set average.

The two preprocessing steps were combined using ColumnTransformer:

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder, StandardScaler

preprocessor = ColumnTransformer(
    transformers=[
        (
            "categorical",
            OneHotEncoder(handle_unknown="ignore"),
            categorical_features,
        ),
        (
            "numeric",
            StandardScaler(),
            numeric_features,
        ),
    ]
)

This says:

apply OneHotEncoder to categorical_features
apply StandardScaler to numeric_features
combine the resulting columns into one model-readable matrix

Appendix A7: Training-Only Preprocessing

Most models used scikit-learn pipelines so that preprocessing was fit only on the training data and then applied unchanged to the temporal holdout.

This matters because preprocessing itself can leak future information.

Suppose the model is evaluated on later opportunities, but the scaler is fit using both earlier and later opportunities. Then the training process has already seen information about the distribution of the future test set.

That gives the model information it would not have had at the time of prediction.

The safer workflow is:

split rows into train and test by time
fit preprocessing on training rows only
transform training rows
train model
transform future test rows using the same fitted preprocessing
evaluate model

The preprocessing step and classifier were wrapped in a Pipeline:

from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression

pipeline = Pipeline(
    steps=[
        ("preprocessor", preprocessor),
        ("classifier", LogisticRegression(max_iter=2000)),
    ]
)

Training and prediction then followed the standard scikit-learn pattern:

pipeline.fit(X_train, y_train)

predicted_probability = pipeline.predict_proba(X_test)[:, 1]

Inside pipeline.fit, scikit-learn does this:

preprocessor.fit_transform(X_train)
classifier.fit(transformed_X_train, y_train)

Inside pipeline.predict or pipeline.predict_proba, it does this:

preprocessor.transform(X_test)
classifier.predict(...) or classifier.predict_proba(...)

Notice that the test data is transformed, but the preprocessing rules are not refit on the test data.

For the next-event prediction experiment, the target was multiclass rather than binary. Those models used an SGDClassifier with logistic loss:

from sklearn.linear_model import SGDClassifier

classifier = SGDClassifier(
    loss="log_loss",
    max_iter=1000,
    tol=1e-3,
    random_state=42,
)

In this setting, the classifier is not asking "won or lost?" It is asking:

which activity is most likely to happen next?

So the prediction is a class label:

prediction = pipeline.predict(X_test)

and the evaluation uses multiclass metrics.

Appendix B1: Evaluation Tasks

The experiments use two kinds of prediction tasks.

The first kind is binary ranking: rank won deals above lost deals, or successful applications above unsuccessful applications.

The second kind is multiclass classification: predict exactly which event happens next.

Those tasks need different metrics.

Appendix B2: ROC AUC

For binary opportunity-outcome experiments, ranking performance was measured with ROC AUC:

from sklearn.metrics import roc_auc_score

auc = roc_auc_score(
    y_test,
    predicted_probability
)

ROC AUC measures how often the model ranks a randomly selected positive case above a randomly selected negative case.

Here is a small worked example.

There are two positive examples, A and C, and two negative examples, B and D.

That creates four positive-negative pairs:

Three of the four pairs are ordered correctly, so:

With ties, AUC gives half credit. The general counting version is:

This is why AUC is a ranking metric. It does not require the predicted probability to be perfectly calibrated. It asks whether positive examples tend to receive higher scores than negative examples.

An AUC of:

is equivalent to random ranking, while:

represents perfect ranking.

In the synthetic experiment, AUC was useful because the observed outcome $Y_i$ was stochastic. Even if a deal had high true probability, it could still fail to close. AUC asks whether the model tends to rank the closed deals higher overall.

Appendix B3: Spearman Rank Correlation

The synthetic experiment also compares the model's predicted ranking against latent quantities that are known only because the data was simulated.

This was measured using Spearman rank correlation:

from scipy.stats import spearmanr

rank_corr, _ = spearmanr(
    predicted_probability,
    test_appetite
)

Spearman correlation first converts values into ranks.

For example:

raw scores:   0.20   0.90   0.40
ranks:          1      3      2

Then it compares the rank ordering of one variable with the rank ordering of another variable.

Here is a worked example with four opportunities.

Without ties, Spearman correlation can be computed as:

In this example:

and:

A value near:

means the two rankings are almost the same.

A value near:

means there is no consistent ranking relationship.

A value near:

means the rankings are almost opposite.

This is why the synthetic experiment can distinguish:

from:

The first compares the model ranking with total latent appetite. The second compares it with the systematic ordering recoverable from observable features.

This distinction matters because:

The model can learn structure in $S_i$ because $S_i$ is built from observable inputs. It cannot directly recover the realized value of $\epsilon_i$ because that term is hidden.

Appendix B4: Accuracy

For next-event prediction, the target has many possible classes, so the experiment reports accuracy, macro F1, and weighted F1.

Accuracy measures the proportion of predictions that are exactly correct.

For example, if the model makes $100$ predictions and gets $84$ right:

In scikit-learn:

from sklearn.metrics import accuracy_score

accuracy = accuracy_score(
    y_test,
    prediction
)

Accuracy is easy to understand, but it can hide poor performance on rare classes.

Suppose a dataset has $100$ next events:

80 are A_Submitted
15 are O_Create Offer
5 are A_Cancelled

A model that mostly predicts the common class can look decent by accuracy while still being bad at rare but important events.

Appendix B5: Precision, Recall, and F1

F1 is built from precision and recall.

For one class, precision asks:

Of the cases the model predicted as this class, how many were actually this class?

Recall asks:

Of the cases that truly belonged to this class, how many did the model find?

For one class:

and:

Then:

A small example for one class:

true positives   = 8
false positives  = 2
false negatives  = 4

Then:

and:

so:

In scikit-learn:

from sklearn.metrics import f1_score

macro_f1 = f1_score(
    y_test,
    prediction,
    average="macro"
)

weighted_f1 = f1_score(
    y_test,
    prediction,
    average="weighted"
)

Appendix B6: Macro F1 and Weighted F1

Macro F1 calculates F1 separately for each class and gives every class equal weight.

Weighted F1 also calculates F1 separately, but weights classes according to how often they occur.

Suppose a next-event model has these class-level F1 scores:

Macro F1 is:

Weighted F1 is:

The same model can therefore have high weighted F1 and much lower macro F1.

That pattern means:

the model is doing much better on common classes than rare classes.

The increase in macro F1:

therefore matters because it suggests that resource context improved prediction across the set of next-event classes, not only for the most frequent transition.

Appendix C1: Data and Reproducibility

The notebooks are published in the companion GitHub repository:

https://github.com/vbnovikov/deal-appetite-experiment

Raw datasets are not included in the repository. They should be downloaded from their original sources and placed under the local data/ directory described in DATA_SOURCES.md.

The repository is licensed under the MIT License. The datasets remain subject to their original source licenses.