[nissanneill]

Fire bricks can vary quite considerably in density and porosity,
If you have refactory cement which dries very hard, or the cheaper fireclay/lime/portland cement, (which also dries hard over a few days), then you may use a softer wire brush on a small angle grinder. Care must be taken to gently ease the deposit off raster that have it gouge out the brick surface.
If you want to return the surface to the best possible under the circumstances, an orbital or random orbital sander can also be used but to lay safe use a fine grit paper (or better cloth abrasive).
These methods will however change the surface appearence but will remove that stubborn cement.

Neill