Replays for React Scan #200

RobPruzan · 2025-01-22T04:12:32Z

This PR Introduces a runtime for react scan to:

collect detailed interaction timing by hacking details from browser event loop callbacks
associate the custom interaction timing tracking with the PerformanceEntry API to derive the time between commit end and new frame presentation
collect rrweb session replay events
runs custom rrweb plugins that:
- collect react-scan data, allowing the render outlines to be replayed without the user seeing them
- collect live fps updates (for the purpose of correlating them to the replay)
- track interaction timings during the session replay

Other changes include:

an rrweb replayer plugin to render a canvas over the session replay with react scan outlines
updated render tracking used in react scan monitoring
- we now only track the blocking component renders
collect the fiber subtree that re-rendered when ingesting in monitoring
also collect the subtree of interacted element, incase the target was not the leaf node

What the PR does not include is the infrastructure behind replays for react scan, that will be open sourced at a future date

That functionaliy roughly includes:

an ingestion server to collect interaction data
a puppeteer cluster to replay the session replay offline, this allows us to:
- convert the session replay to an mp4 by taking successive screenshots of the replayed session
- take screenshots of the element interacted with
- take screenshots of the render outlines over the page when the interaction happened
- run expensive dom queries to understand the state of the app that we can't run on the users device
  - for example, querying for every single dom element if its in the viewport. This is not feasible to run on user device, but is trival to run offline in puppeteer
storage for replays as mp4's, element screenshots, and video thumbnails
Interaction summary w/ LLM's without using personalized user data
- we never send data about the state of the application to a third party. This means we are limited in how good the summaries can get using only "static" information about the fiber tree. But, with a good enough prompt, the model does a good job in the general case

Preview of visualized collected data

demo-video-3-compressed.mp4

vercel · 2025-01-22T04:12:37Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
react-scan	❌ Failed (Inspect)			Jan 22, 2025 7:41am

RobPruzan · 2025-01-22T04:34:46Z

packages/scan/package.json

@@ -1,8 +1,14 @@
 {
  "name": "react-scan",
-  "version": "0.0.54",
+  "version": "0.0.1083",


this was incremented every time i tested a new change...

RobPruzan · 2025-01-22T04:35:35Z

packages/scan/src/core/monitor/performance.ts

+ *
+ * handles tracking event timings for arbitrarily overlapping handlers with cancel logic
+ */
+export const setupDetailedPointerTimingListener = (


where the magic happens

lxsmnsyc · 2025-01-22T04:40:59Z

packages/scan/src/core/index.ts

+      !domFiber ||
+      !domFiber.stateNode ||
+      !(domFiber.stateNode instanceof Element)
+    )


De Morgan's Law 😉

lxsmnsyc · 2025-01-22T04:46:04Z

packages/scan/src/core/monitor/session-replay/record.ts

+};
+
+export const startNewRecording = async () => {
+  console.log('new recording start');


this PR is not in a reviewable state yet (there are probably on the order of 1000s comments that could be left)

only opening it for visibility

lxsmnsyc · 2025-01-22T04:46:32Z

packages/scan/src/core/monitor/session-replay/record.ts

+
+let lastFiberId = 0;
+const fiberIdMap = new Map<Fiber, number>();
+const getFiberId = (fiber: Fiber) => {


how different is this from bippy's getFiberID

i think i wrote that before it existed

lxsmnsyc · 2025-01-22T04:47:17Z

packages/scan/src/core/monitor/session-replay/record.ts

+  //   "first rrweb event must be meta event"
+  // );
+  if (payload.events[0].type !== 4) {
+    console.warn('hm', payload);


we probably need a DEV flag here now lol, perhaps import.meta.env.DEV

collect detailed browser timing on interaction implement react scan rrweb replayer plugin refactor monitoring interaction and component collection

imcodingideas · 2025-01-23T22:32:12Z

packages/scan/src/index.ts

@@ -1,3 +1,5 @@
 import 'bippy'; // implicit init RDT hook

 export * from './core/index';
+
+export { ReactScanReplayPlugin } from './core//monitor/session-replay/replay-v2';


Should we remove the extra / here?

imcodingideas · 2025-01-23T22:39:46Z

packages/scan/src/core/monitor/performance-store.ts

+  channels: Record<
+    ChanelName,
+    { callbacks: BoundedArray<Callback>; state: BoundedArray<Item> }
+  >;


I think you can do something like this, and use generics here.

Suggested change

channels: Record<

ChanelName,

{ callbacks: BoundedArray<Callback>; state: BoundedArray<Item> }

>;

channels: Record<ChannelName, Channel<T>>;

and you could create an interface for it like this..

interface Channel<T> { callbacks: BoundedArray<Callback<T>>; state: BoundedArray<T>; }

imcodingideas · 2025-01-23T22:46:28Z

packages/scan/src/core/monitor/performance-store.ts

+type PerformanceEntryChannelsType = {
+  subscribe: (to: ChanelName, cb: Callback) => UnSubscribe;
+  publish: (
+    item: Item,
+    to: ChanelName,
+    dropFirst: boolean,
+    createIfNoChannel: boolean,
+  ) => void;
+  channels: Record<
+    ChanelName,
+    { callbacks: BoundedArray<Callback>; state: BoundedArray<Item> }
+  >;
+  getAvailableChannels: () => BoundedArray<string>;
+  updateChannelState: (
+    channel: ChanelName,
+    updater: Updater,
+    createIfNoChannel: boolean,
+  ) => void;
+};


This is a little more strongly typed here.. and you can use the generics when you implement PerformanceEntryChannelsType

Suggested change

type PerformanceEntryChannelsType = {

subscribe: (to: ChanelName, cb: Callback) => UnSubscribe;

publish: (

item: Item,

to: ChanelName,

dropFirst: boolean,

createIfNoChannel: boolean,

) => void;

channels: Record<

ChanelName,

{ callbacks: BoundedArray<Callback>; state: BoundedArray<Item> }

>;

getAvailableChannels: () => BoundedArray<string>;

updateChannelState: (

channel: ChanelName,

updater: Updater,

createIfNoChannel: boolean,

) => void;

};

export type ChannelItem = InternalInteraction | PerformanceInteraction;

export type UnSubscribe = () => void;

export type Callback<T> = (item: T) => void;

export type Updater<T> = (state: BoundedArray<T>) => BoundedArray<T>;

export type ChannelName = string;

interface Channel<T> {

callbacks: BoundedArray<Callback<T>>;

state: BoundedArray<T>;

}

interface PerformanceEntryChannelsType<T> {

subscribe: (to: ChannelName, cb: Callback<T>, dropFirst?: boolean) => UnSubscribe;

publish: (

item: T,

to: ChannelName,

createIfNoChannel?: boolean

) => void;

channels: Record<ChannelName, Channel<T>>;

getAvailableChannels: () => BoundedArray<string>;

updateChannelState: (

channel: ChannelName,

updater: Updater<T>,

createIfNoChannel?: boolean

) => void;

getChannelState: (channel: ChannelName) => BoundedArray<T>;

}

imcodingideas · 2025-01-23T22:54:27Z

packages/scan/src/core/monitor/performance-store.ts

+class PerformanceEntryChannels implements PerformanceEntryChannelsType {
+  channels: PerformanceEntryChannelsType['channels'] = {};
+  publish(item: Item, to: ChanelName, createIfNoChannel = true) {
+    const existingChannel = this.channels[to];
+    if (!existingChannel) {
+      if (!createIfNoChannel) {
+        return;
+      }
+      this.channels[to] = {
+        callbacks: new BoundedArray<Callback>(MAX_CHANNEL_SIZE),
+        state: new BoundedArray<Item>(MAX_CHANNEL_SIZE),
+      };
+      this.channels[to].state.push(item);
+      return;
+    }
+
+    existingChannel.state.push(item);
+    existingChannel.callbacks.forEach((cb) => cb(item));
+  }
+
+  getAvailableChannels() {
+    return BoundedArray.fromArray(Object.keys(this.channels), MAX_CHANNEL_SIZE);
+  }
+  subscribe(to: ChanelName, cb: Callback, dropFirst: boolean = false) {
+    const defer = () => {
+      if (!dropFirst) {
+        this.channels[to].state.forEach((item) => {
+          cb(item);
+        });
+      }
+      return () => {
+        const filtered = this.channels[to].callbacks.filter(
+          (subscribed) => subscribed !== cb
+        );
+        this.channels[to].callbacks = BoundedArray.fromArray(filtered, MAX_CHANNEL_SIZE);
+      };
+    };
+    const existing = this.channels[to];
+    if (!existing) {
+      this.channels[to] = {
+        callbacks: new BoundedArray<Callback>(MAX_CHANNEL_SIZE),
+        state: new BoundedArray<Item>(MAX_CHANNEL_SIZE),
+      };
+      this.channels[to].callbacks.push(cb);
+      return defer();
+    }
+
+    existing.callbacks.push(cb);
+    return defer();
+  }
+  updateChannelState(
+    channel: ChanelName,
+    updater: Updater,
+    createIfNoChannel = true,
+  ) {
+    const existingChannel = this.channels[channel];
+    if (!existingChannel) {
+      if (!createIfNoChannel) {
+        return;
+      }
+
+      const state = new BoundedArray<Item>(MAX_CHANNEL_SIZE)
+      const newChannel = { callbacks: new BoundedArray<Item>(MAX_CHANNEL_SIZE), state };
+
+      this.channels[channel] = newChannel;
+      newChannel.state = updater(state);
+      return;
+    }
+
+    existingChannel.state = updater(existingChannel.state);
+  }
+
+  getChannelState(channel: ChanelName) {
+    return this.channels[channel].state ?? new BoundedArray<Item>(MAX_CHANNEL_SIZE);
+  }
+}
+
+export const performanceEntryChannels = new PerformanceEntryChannels();


This is really great. I'm so excited for this feature.

Fly on the wall here without write access. Happy to help with TypeScript a little I don't step on your toes.

Suggested change

class PerformanceEntryChannels implements PerformanceEntryChannelsType {

channels: PerformanceEntryChannelsType['channels'] = {};

publish(item: Item, to: ChanelName, createIfNoChannel = true) {

const existingChannel = this.channels[to];

if (!existingChannel) {

if (!createIfNoChannel) {

return;

}

this.channels[to] = {

callbacks: new BoundedArray<Callback>(MAX_CHANNEL_SIZE),

state: new BoundedArray<Item>(MAX_CHANNEL_SIZE),

};

this.channels[to].state.push(item);

return;

}

existingChannel.state.push(item);

existingChannel.callbacks.forEach((cb) => cb(item));

}

getAvailableChannels() {

return BoundedArray.fromArray(Object.keys(this.channels), MAX_CHANNEL_SIZE);

}

subscribe(to: ChanelName, cb: Callback, dropFirst: boolean = false) {

const defer = () => {

if (!dropFirst) {

this.channels[to].state.forEach((item) => {

cb(item);

});

}

return () => {

const filtered = this.channels[to].callbacks.filter(

(subscribed) => subscribed !== cb

);

this.channels[to].callbacks = BoundedArray.fromArray(filtered, MAX_CHANNEL_SIZE);

};

};

const existing = this.channels[to];

if (!existing) {

this.channels[to] = {

callbacks: new BoundedArray<Callback>(MAX_CHANNEL_SIZE),

state: new BoundedArray<Item>(MAX_CHANNEL_SIZE),

};

this.channels[to].callbacks.push(cb);

return defer();

}

existing.callbacks.push(cb);

return defer();

}

updateChannelState(

channel: ChanelName,

updater: Updater,

createIfNoChannel = true,

) {

const existingChannel = this.channels[channel];

if (!existingChannel) {

if (!createIfNoChannel) {

return;

}

const state = new BoundedArray<Item>(MAX_CHANNEL_SIZE)

const newChannel = { callbacks: new BoundedArray<Item>(MAX_CHANNEL_SIZE), state };

this.channels[channel] = newChannel;

newChannel.state = updater(state);

return;

}

existingChannel.state = updater(existingChannel.state);

}

getChannelState(channel: ChanelName) {

return this.channels[channel].state ?? new BoundedArray<Item>(MAX_CHANNEL_SIZE);

}

}

export const performanceEntryChannels = new PerformanceEntryChannels();

class PerformanceEntryChannels<T> implements PerformanceEntryChannelsType<T> {

private readonly channels: Record<ChannelName, Channel<T>> = {};

private createChannel(): Channel<T> {

return {

callbacks: new BoundedArray<Callback<T>>(MAX_CHANNEL_SIZE),

state: new BoundedArray<T>(MAX_CHANNEL_SIZE),

};

}

private ensureChannel(channelName: ChannelName, createIfNoChannel = true): Channel<T> | null {

if (!this.channels[channelName]) {

if (!createIfNoChannel) {

return null;

}

this.channels[channelName] = this.createChannel();

}

return this.channels[channelName];

}

public publish(item: T, to: ChannelName, createIfNoChannel = true): void {

const channel = this.ensureChannel(to, createIfNoChannel);

if (!channel) return;

channel.state.push(item);

channel.callbacks.forEach((cb) => {

try {

cb(item);

} catch (error) {

console.error(`Error in channel "${to}" callback:`, error);

}

});

}

public subscribe(to: ChannelName, cb: Callback<T>, dropFirst = false): UnSubscribe {

const channel = this.ensureChannel(to, true);

if (!channel) throw new Error(`Failed to create/access channel: ${to}`);

channel.callbacks.push(cb);

if (!dropFirst) {

channel.state.forEach((item) => {

try {

cb(item);

} catch (error) {

console.error(`Error in channel "${to}" subscription callback:`, error);

}

});

}

return () => {

const filtered = channel.callbacks.filter(

(subscribed) => subscribed !== cb

);

channel.callbacks = BoundedArray.fromArray(filtered, MAX_CHANNEL_SIZE);

};

}

public getAvailableChannels(): BoundedArray<string> {

return BoundedArray.fromArray(Object.keys(this.channels), MAX_CHANNEL_SIZE);

}

public updateChannelState(

channelName: ChannelName,

updater: Updater<T>,

createIfNoChannel = true

): void {

const channel = this.ensureChannel(channelName, createIfNoChannel);

if (!channel) return;

try {

channel.state = updater(channel.state);

} catch (error) {

console.error(`Error updating channel "${channelName}" state:`, error);

}

}

public getChannelState(channelName: ChannelName): BoundedArray<T> {

const channel = this.channels[channelName];

return channel?.state ?? new BoundedArray<T>(MAX_CHANNEL_SIZE);

}

public clearChannel(channelName: ChannelName): void {

const channel = this.channels[channelName];

if (channel) {

channel.state = new BoundedArray<T>(MAX_CHANNEL_SIZE);

}

}

public removeChannel(channelName: ChannelName): void {

delete this.channels[channelName];

}

public getSubscriberCount(channelName: ChannelName): number {

return this.channels[channelName]?.callbacks.length ?? 0;

}

}

thanks for the contributions, for context I will probably be migrating this logic over for a react-scan feature (some harder backend problems preventing this from being immediately implemented)

whatever changes i bring over, ill do my best to give attribution :)

add monitor debugging scripts

f982809

RobPruzan changed the title ~~Replay for React Scan~~ Replays for React Scan Jan 22, 2025

RobPruzan commented Jan 22, 2025

View reviewed changes

lxsmnsyc reviewed Jan 22, 2025

View reviewed changes

RobPruzan force-pushed the monitoring-session-replay branch from f4de562 to 90328f3 Compare January 22, 2025 07:38

vercel bot had a problem deploying to Preview January 22, 2025 07:38 Failure

RobPruzan force-pushed the monitoring-session-replay branch 5 times, most recently from 818ac8d to c6456e9 Compare January 22, 2025 07:40

feat(scan): replay collection runtime

fd0b84e

collect detailed browser timing on interaction implement react scan rrweb replayer plugin refactor monitoring interaction and component collection

RobPruzan force-pushed the monitoring-session-replay branch from c6456e9 to fd0b84e Compare January 22, 2025 07:41

vercel bot had a problem deploying to Preview January 22, 2025 07:41 Failure

imcodingideas reviewed Jan 23, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replays for React Scan #200

Replays for React Scan #200

RobPruzan commented Jan 22, 2025 •

edited

Loading

vercel bot commented Jan 22, 2025 •

edited

Loading

RobPruzan Jan 22, 2025

RobPruzan Jan 22, 2025

lxsmnsyc Jan 22, 2025

lxsmnsyc Jan 22, 2025

RobPruzan Jan 22, 2025 •

edited

Loading

lxsmnsyc Jan 22, 2025

RobPruzan Jan 22, 2025

lxsmnsyc Jan 22, 2025

imcodingideas Jan 23, 2025

imcodingideas Jan 23, 2025 •

edited

Loading

imcodingideas Jan 23, 2025 •

edited

Loading

imcodingideas Jan 23, 2025

RobPruzan Jan 26, 2025

Replays for React Scan #200

Are you sure you want to change the base?

Replays for React Scan #200

Conversation

RobPruzan commented Jan 22, 2025 • edited Loading

Preview of visualized collected data

vercel bot commented Jan 22, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RobPruzan Jan 22, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

imcodingideas Jan 23, 2025 • edited Loading

Choose a reason for hiding this comment

imcodingideas Jan 23, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RobPruzan commented Jan 22, 2025 •

edited

Loading

vercel bot commented Jan 22, 2025 •

edited

Loading

RobPruzan Jan 22, 2025 •

edited

Loading

imcodingideas Jan 23, 2025 •

edited

Loading

imcodingideas Jan 23, 2025 •

edited

Loading