That will affect pedal height.
Pedal feel, hardness like you're dealing with is another matter.
My experience has shown me that a hard pedal with poor braking is a matter of
not having enough leverage.
You can have poor pedal leverage and the force your foot makes is not leveraged/amplified enough to generate adequate force to the calipers or drums.
The booster systems have less leverage than a manual brake system but the booster amplifies the amount of pressure your foot gives the pedal.
With my setup, I tried numerous master cylinders when I tried to run manual brakes. I drilled a hole higher up on the pedal to increase the leverage. It still wasn't enough. For me, I needed additional boost to reduce the pedal effort to comfortably stop the car. If I had Hercules legs, the manual setup may have been fine if it didn't bend the linkage doing so.
Even with the first version of power brakes, it wasn't optimal since I was using an A body single diaphragm disc/drum unit. That worked fine when I had rear drums but the additional volume needed when I switched to the rear discs made the old booster inadequate.
In short, it sounds like you need to look at the booster. In the two examples I have experience with recently, both of these cars:
Have stock reproduction 1966-70 B body disc/drum dual diaphragm boosters, stock linkage parts and the pedal feel is excellent with both.
The red car does have the adjustable pushrod:
The pushrod did not affect the pedal feel, it was merely cosmetic.....to make the brake pedal rest at the same height as the clutch pedal.