Hi,
I have the exact problem with my BMW (P0031, P0051, P0037, P0057 Errors).
The fuse did blow every time I started the car. When I disconnect the Angle Connector the fuse didn't blow. Now I have replaced the Angle Connector and the fuse is fine but the engine light wont go away. I have tried to clear the codes with my ODB2 and for the moment the codes and engine light disappears (I can drive with the light of but when I stop the engine and restart the car again the engine light and the same codes come back).... The fuse is fine. Before I replaced the Angle Connector the dealer checked every O2 sensor and they were all good.
Maybe NOT the exact CAUSE of the similar symptoms or identical codes.

If you would post your model year, Chassis#, Engine, etc. (e.g. 2007 328xi, N52KP) someone could provide you with the electrical schematic for the O2 Sensor heater circuit, which includes the Angle Connector (AKA PCV Breather Heater

and we might be able to figure out why the codes appear on restart, even though the Heater/Angle Connector and fuse have been replaced and are fine. If you want to learn how to use TIS to get that information yourself, just use this link and enter your vehicle VIN or other identifying info:
https://www.newtis.info/tisv2/a/en/
One of the problems arising from posting to an OLD thread (last post prior to yours was October, 2016) is that "Stuff Happens."
In this case, the "Stuff" is Recall Campaign # 17V-683, related to the "Breather Heater" or Angle Connector on all 328i/xi models 2007 to 2011, which was announced in October 2017. BMW has been VERY slow to provide a fix, make parts available, and contact the owners of such availability and to schedule a Dealer visit (I'm registered with BMW NA, received the first form letter in December, but have NOT received ANY further communication).
HOWEVER, if you were to call your nearest Dealer, they SHOULD offer an inspection to try to determine the fault in your case. However, do NOT "bet the ranch" on the idea that BMW dealers have the ability, based upon the personal experience of whatever tech you draw, to identify or find the problem in your particular case, for the reasons suggested below.
Keep in mind the codes you are getting are for low voltage (possibly/probably NONE) in each of the O2 sensor heater circuits. That is what happens when a short in the Breather Heater (in parallel on the same fuse circuit as O2 Sensors) blows the fuse -- power is also lost to EACH of the 4 O2 Sensor Heaters. Here is the TIS circuit diagram for my 2007 328xi:
https://www.newtis.info/tisv2/a/en/.../e91-328xi-tou_200703/components-connectors/components/f-fuses/f79-fuse/vEO0UYq
Do NOT assume that the wiring diagram is identical for YOUR vehicle. Find the diagram for YOUR vehicle or post data (such as last 7 of VIN) for your vehicle and I'll give you the link to the circuit diagram (however it WOULD benefit YOU to take the time to learn how to navigate TIS ;-)
In looking at that Schematic, nothing jumps off the page at me to suggest WHY no code (or at least SES light) during operation, but it returns on restart. Perhaps the way the warning light or code memory process is "programmed" is that even when you have NO voltage to the O2 sensor heaters (test at ANY of the Orange Wires), the SES does NOT light until the next drive cycle, and even though you Clear the codes, there is still NO voltage. There are of course other possibilities, such as a damaged DME monitoring circuit, but rather than worry about the worst possible scenario, there ARE possible/likely scenarios that YOU can test for yourself, with the benefit of TIS Schematics.
Your issue COULD be due to the fact that wiring was damaged at the time of the Breather Heater short and blowing of the fuse, and although you appear to be assuming that replacing the Breather Heater/Angle Connector and fuse corrected the problem, it ONLY corrected the problem of a blown fuse. There appears to STILL be an open circuit downstream from that fuse which is setting the P0031 etc. O2 Heater Voltage Errors.
Note that in the attached Schematic, the Breather Heater is at the END of the parallel wiring chain, or at least supplied by the same connector (X6961) as the O2 Sensor Heaters, so wire damage at ANY point between the fuse and Breather Heater is possible if the fuse did NOT blow fast enough.
If you have INPA, that SHOULD tell you if you have a fault, even if it has NOT yet set a warning lamp (SES light). I'm NOT sure if a generic code reader will do the same. Problem is that "Monitors" that are governmentally-required for purposes of setting the SES (and code) if there is a failure in the Heater Circuit, may NOT light the SES until the start of the next drive cycle, and AFAIK, there is NO function in INPA that provides a readout of the Heater status or voltage, ONLY a Monitor Readiness Status for the Heaters (which to me suggests a Drive Cycle warning lamp scenario). INPA, or many generic scan tools, CAN of course provide real-time readout of Sensor signal voltage, but that is NOT heater voltage.
All the code software theorizing (SWAGs

aside, I would suggest tracing the wiring from the fuse to the O2 Heaters. Actually, if wiring damage creating an open circuit IS the cause of your issue, then as I read the Schematic, only ONE test is needed to confirm that:
1) Look at X6961 on the Schematic, and you will see THAT is the common Junction (like a splice) for ALL 5 components powered by the fuse (4 O2 sensors and the Breather Heater). So if you have NO voltage at X6961 with ignition on, then there is a wiring fault between X6961 and the fuse. Also, in the case of a short, heat damage is often found in the area of relevant pin location in a connector, so the damaged area may be readily apparent upon careful examination of one of more of the connectors, without having to inspect the full run of the relevant wire itself.
So how do you find X6961 to be able to test?
TIS Schematics are interactive, meaning you can click on any of the BLUE component ID#s and get more information on that component. In this case, if you click on X6961 and select "Installation Location" you get this diagram:
https://www.newtis.info/tisv2/a/en/e91-328xi-tou_200703/S8C692L
Likewise, clicking on either of the two Connectors, X60551 or X6011 and selecting Installation Location, provides this diagram which shows the location of both connectors:
https://www.newtis.info/tisv2/a/en/e91-328xi-tou_200703/SL92ADs
So MOST of the wiring, and ALL of the connectors you need to check are in the E-box.
Please let us know what you find,
George