There is a combination of apps that should work. If you combine Easy Voice Recorder, Saiy and Tasker, you should be able to get the desired result.
Saiy is a voice assistant that can replace Google Voice Assistant. Most importantly, it is compatible with the Gemini Voice Assistant (in that you can pick it to be the default Voice Assistant when you push the side button). It also integrates with Tasker as a plugin
Easy Voice Recorder is an app that (believe it or not) record your voice. It also integrates with Tasker
Tasker is the app that would tie this all together. Basically, you would set Saiy to trigger a Tasker task that you make (let's call it "Recording"). That task would toggle the Easy Voice Recorder on or off. In theory, you could hit the silver button and start recording. Even better, you could set a custom voice trigger. So you could hit the silver button, and say "Start Recording". That way you can also still use the silver button for other voice tasks.
I haven't tested this, but I'm pretty sure it would work. If you want a step-by-step let me know. Oh - the only thing to keep in mind is Takser isn't free. but it is well worth the price considering all the cool things you can do with it and the Gemini!
Oh - lastly - this is on Oreo. No idea if Saiy still works on Pie...