The switch would probably make sense if coupled with a heat store in order to take advantage of night rates (Economy 7, e.g.) but for that you need space, a boiler room of sorts, heat inputs, volumes, cabling, etc. would need to be calculated as per expected demand for heat and hot water. You might want to consider measuring and charging your tenants only for the heat and hot water they will be using, energy / flow meters can be linked to an automated invoicing / reporting system.
You might, also, consider a heat pump instead of an electric boiler if there is land adjacent to the property, or loft that can be sound-proofed from the flats below, or solar panels if the roof is southerly oriented.
Costs and benefits of the solutions can be calculated with a reasonable precision.

Is the Oil doing all of the flats and is the flats individual let? If so, I would individually put in their own Heating / Hot Water. So each Tenant will have there own charge and used by electric.

Remove the Oil Equipment, go for something like a Heat Rae Sadia Electomax. The outlay will be expensive, but once all installed, the running cost will be down to the Tenant. I presume you want individual control over the 6 flats and they independently control their own Heating / Hot Water?

Or just 1 Electromax will do the property if you go for a plant room. But the disadvantage, no independent control over the Heating / Hot Water. But the change can happen from Oil To other source of Energy.

TY