MRR
Mean Reciprocal Rank rewards putting a hidden liked story near the top of the ranked list.
TECHNICAL TRANSPARENCY
Methodology
This page separates what RR Signal has validated from what is still exploratory. It is intentionally more technical than the plain-English guide.
The behavioral graph
RR Signal represents public Favorites as a sparse reader×story graph. A reader row contains the stories observed in that reader's qualifying public Favorite history. A story column is connected to the readers who favored it.
Recruitment stories used to discover evaluation readers are removed from benchmark features and targets so the system cannot receive credit for information that was already used to find those readers.
Current recommender
The validated incumbent uses BM25-style weighting followed by a low-rank SVD representation. BM25 reduces the influence of extremely common stories and normalizes for reader Favorite-list depth. SVD then learns a small set of latent behavioral dimensions from the sparse graph.
Latent rank8
BM25 k11.2
BM25 b0.5
The live reader recommender blends the latent model with popularity according to profile depth: shallow profiles rely more on popularity; deeper profiles rely much more heavily on personalization.
What has been validated
In held-out recommendation tests, the fixed collaborative recommender beat global popularity on the research cohorts. The strongest confirmation work used readers with enough non-recruitment Favorites to hide targets while leaving at least 10 known stories.
The validated endpoint is reader recommendation ranking. That does not automatically mean every
story-to-story Author Intelligence feature has the same evidence status.
How ranking quality is measured
NDCG@10
Measures whether relevant hidden stories appear high in the first ten recommendations.
RR Signal uses paired reader-level bootstrap confidence intervals when comparing models, so the comparison is based on differences for the same evaluation readers rather than unrelated aggregate averages.
Author Intelligence evidence status
Author Intelligence uses the same behavioral representation plus direct shared-reader evidence, but True Comps, Audience Gaps and Collaboration Targets are product hypotheses layered on top of that graph.
They are therefore labeled experimental. RR Signal evidence-gates sparse stories and withholds collaboration recommendations when direct support is too thin.
Current graph vs. Phase 5
The live product currently serves a frozen historical research graph. A larger Phase 5 graph is being collected separately and is not silently feeding the live model.
When Phase 5 reaches its target, collection will be stopped, the benchmark will be frozen and hashed, and the expanded incumbent will be compared against the current baseline under a predeclared decision rule. Challenger model families are selected on validation before confirmatory test comparison.
What RR Signal intentionally avoids
- No claim that model scores are probabilities of liking a story.
- No invented “reader tribe” labels presented as behavioral fact.
- No use of recruitment anchors as evidence in the benchmark that recruited those readers.
- No chapter-text corpus is required for the behavioral recommender.
- No claim that a behavioral relationship proves causal influence between stories or authors.