[bounty] $1000 make RDP work

[louis030195]

current status:

i did a large part of the work here:
#686

but something is not working:

screenshots are showing my own screen instead of the connected users' screen

i properly am able to list sessions, get tokens, etc. but then it fails - tried both in python and in rust, same issue

definition of done:

make one of those bins work

https://github.com/mediar-ai/screenpipe/pull/686/files#diff-5f59a0279ffec923a1f389f4ec443da49d2e0bcf4de83d0f154a0a8ccf5d4e16

https://github.com/mediar-ai/screenpipe/pull/686/files#diff-62d7d5d81348edcf32b86ef4546e79b86bc50b7c1fa580f011426c5acc4f9569

eg:

• screenpipe runs on a Windows 22 server
• n clients connect to it through Microsoft Remote Desktop, each seeing their own user/session/screen
• screenpipe capture each client screen/session (and not the local server screen)

you can send a new PR if you want - copy pasting my code and making it work

i can do the plumbing to the rest of the code (which is the easy part)

PS: it's a screenpipe enterprise solution, not related to our main codebase

/bounty 1000

[algora-pbc]

💎 $1,000 bounty • Screenpi.pe

Steps to solve:

1. Start working: Comment /attempt #850 with your implementation plan
2. Submit work: Create a pull request including /claim #850 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to mediar-ai/screenpipe!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @nikhilxe	Dec 6, 2024, 1:17:26 PM	WIP
🟢 @gerred	Jan 2, 2025, 11:35:52 PM	WIP

[Neptune650]

current status:

i did a large part of the work here: #686

but something is not working:

screenshots are showing my own screen instead of the connected users' screen

i properly am able to list sessions, get tokens, etc. but then it fails - tried both in python and in rust, same issue

definition of done:

make one of those bins work

https://github.com/mediar-ai/screenpipe/pull/686/files#diff-5f59a0279ffec923a1f389f4ec443da49d2e0bcf4de83d0f154a0a8ccf5d4e16

https://github.com/mediar-ai/screenpipe/pull/686/files#diff-62d7d5d81348edcf32b86ef4546e79b86bc50b7c1fa580f011426c5acc4f9569

eg:

* screenpipe runs on a Windows 22 server

* n clients connect to it through Microsoft Remote Desktop, each seeing their own user/session/screen

* screenpipe capture each client screen/session (and not the local server screen)

you can send a new PR if you want - copy pasting my code and making it work

i can do the plumbing to the rest of the code (which is the easy part)

PS: it's a screenpipe enterprise solution, not related to our main codebase

/bounty 1000

@louis030195 So if I understand correctly, one of the 3 users connects to the RDP server but does whatever work on their local machine, and screenpipe in the server will capture what each user is doing in their local machine and register it separately.

Doesn't this mean the server will need to be able to see the client's screen? Then wouldn't we need to do it the other way around? I.e. the server connects through RDP to the user machines, and captures the RDP contents.

[louis030195]

@Neptune650 there is supposedly an API to just get the stream of user screen

because most of the time cloud desktop, like cloud gaming, etc. is just streaming the server OS to the local computer

[Neptune650]

@Neptune650 there is supposedly an API to just get the stream of user screen

because most of the time cloud desktop, like cloud gaming, etc. is just streaming the server OS to the local computer

@louis030195 btw, the "rdp2" program and screenpipe both run on the server? Or rdp2 runs on the client? We can talk over telegram if you'd like.

[nikhilxe]

/attempt #850

Algora profile	Completed bounties	Tech	Active attempts	Options
@nikhilxe	1 bounty from 1 project	C++, JavaScript, CSS & more		Cancel attempt

[Antony007]

@louis030195 I don't think it would be straight-forward due to Windows restrictions.

My only guess to achieve this is, we need to add run-as-service feature to the app, then run/install rdp2.exe as a service eg. rdp2.exe --service. Listen to user logon events, and on every new user logon, execute itself i.e rdp2.exe again, but in the respective user session context.

So there will be one process running as service and then one process each for every user session.

rdp2.exe --service (Service)
|_ rdp2.exe (User 1)
|_ rdp2.exe (User 2)
|_ rdp2.exe (User 3)

This approach should definitely work

PS: Also need to ensure, rdp2.exe is in a Path that is accessible to other users as well, like %PROGRAMFILES%, and have execute permission for user group.

[louis030195]

@Neptune650 there is supposedly an API to just get the stream of user screen
because most of the time cloud desktop, like cloud gaming, etc. is just streaming the server OS to the local computer

@louis030195 btw, the "rdp2" program and screenpipe both run on the server? Or rdp2 runs on the client? We can talk over telegram if you'd like.

yes i just run rdp2 or rdp on the server

https://screen-record.com/ is able to record users' sessions without having to do anything on the client side

other thing to try https://gist.github.com/louis030195/e17d4b3c637928b89358c6816e461150

[Antony007]

other thing to try https://gist.github.com/louis030195/e17d4b3c637928b89358c6816e461150

@louis030195 This solution most probably won't work from my experience¹, since RDP itself uses Windows Desktop Duplication API behind the scenes, and it takes exclusive control, preventing other applications from accessing the duplication API while the RDP session is active.

¹ Unless they have changed this behaviour in the latest Windows updates.

[zelosleone]

other thing to try https://gist.github.com/louis030195/e17d4b3c637928b89358c6816e461150

@louis030195 This solution most probably won't work from my experience¹, since RDP itself uses Windows Desktop Duplication API behind the scenes, and it takes exclusive control, preventing other applications from accessing the duplication API while the RDP session is active.

¹ Unless they have changed this behaviour in the latest Windows updates.

@louis030195 Antony is right; to actually record the user’s sessions, you would need a service in users’ client. Honestly, we can try to use fibers for that purpose to not be detected by kernel, thus not showing up in the process window. Fibers are a legit security issue in the Windows design part, but it can be used like this. But it needs to be attached to a process to work. So, I am not sure if it's possible with sandboxing of modern applications, and this goes straight into malware territory. But I think they just record the user by not showing themselves in the process manager using a similar method, then send the data in-real time.

To record multiple remote desktop sessions on a server like Windows Server 2022, you can capture each session's screen individually. This is done using the server's built-in tools to record each user's desktop separately. Thus, the cloud gaming.

[zelosleone]

@Neptune650 there is supposedly an API to just get the stream of user screen
because most of the time cloud desktop, like cloud gaming, etc. is just streaming the server OS to the local computer

@louis030195 btw, the "rdp2" program and screenpipe both run on the server? Or rdp2 runs on the client? We can talk over telegram if you'd like.

yes i just run rdp2 or rdp on the server

https://screen-record.com/ is able to record users' sessions without having to do anything on the client side

other thing to try https://gist.github.com/louis030195/e17d4b3c637928b89358c6816e461150

(*dxgi_factory).EnumAdapters1(0, &mut adapter);
if adapter.is_null() {
    return Err("Failed to find DXGI Adapter".into());
}

You are trying to access the wrong driver here. You need to use Microsoft's "Remote Display" driver instead. It wouldn't make sense to try to reach the physical driver of another device from your device.

return Err("Failed to duplicate output".into());

Duplication is not enabled by default for remote displays. Because simply, there is no "real" desktop to capture, and since it will 100% of the time fail, it's going to return your own physical desktop because of AcquireNextFrame

I wasn't sure if I should /-attempt on this, as someone else already started. However, I'll provide feedback anyways.

[louis030195]

@zelosleone not sure to understand,

To record multiple remote desktop sessions on a server like Windows Server 2022, you can capture each session's screen individually. This is done using the server's built-in tools to record each user's desktop separately. Thus, the cloud gaming.

what do you mean by that? which API is used in this case?

[zelosleone]

@zelosleone not sure to understand,

To record multiple remote desktop sessions on a server like Windows Server 2022, you can capture each session's screen individually. This is done using the server's built-in tools to record each user's desktop separately. Thus, the cloud gaming.

what do you mean by that? which API is used in this case?

https://learn.microsoft.com/en-us/windows/win32/termserv/terminal-services-api
WTSVirtualChannelQuery and WTSQuerySessionInformationW via Remote Desktop Services API

[louis030195]

@zelosleone that's exactly what i did first but it was only capturing local machine instead of users:

https://github.com/mediar-ai/screenpipe/pull/686/files#diff-5f59a0279ffec923a1f389f4ec443da49d2e0bcf4de83d0f154a0a8ccf5d4e16

[9bany]

/attempt

[gerred]

/attempt #850

Options

• Cancel my attempt