PixInsight Weighted Batch Preprocessing Script Released

[fatboy1271]

It appears I missed the release of PixInsight 1.8.8 on 2019-11-08. I installed it today and then I had about 3 rounds of updates afterwards. It looks like one of those updates includes a Weighted Batch Preprocessing Script! I'm reading about it now and I'm quite excited about it. Here is what the email says about it:

We are glad to announce the release of Weighted Batch Preprocessing (WBPP), a new development project led by Roberto Sartori and Tommaso Rubechi. WBPP is a much-evolved version of the classical Batch Preprocessing script (BPP), based on original source code written by Kai Wiechen back in 2012. WBPP is now available as a regular update to version 1.8.8-1 of PixInsight on all supported platforms.

For complete information on WBPP, see the tutorial written by the authors, as well as a description document by Roberto Sartori. See also the announcement post on PixInsight Forum.

[fatboy1271]

In the tutorial link above it has a link to download WBPP; however, it looks like it's now been integrated directly into PI. I closed out of PI and when I reopened it, before installing the downloaded version, there were more updates. Now I can choose WBPP from the Scripts menu.

[UlteriorModem]

Thanks yea I had heard about this. Is the idea it can find and reject 'bad' frames?

[fatboy1271]

Thanks yea I had heard about this. Is the idea it can find and reject 'bad' frames?

Don't quote me on this but I believe it's more of a weighting (think of it as a rating system, sort of) on how "good" of an image is and which are better than others. It then orders them from "good to bad..." It can also select the best reference file to use, based on that weighting.

[UlteriorModem]

Ah I see. I have had a chance to try it out. Took a few tries but I got it working. But got an odd error when I went to stack the results!

PI Error by Tom Whit, on Flickr

Thing is though it stacked just fine with good results!

[fatboy1271]

It seems to do a heck of a job! My Diagnostics complained about my Lights and Darks. They are all 180sec, but the Properties show Lights as 180.3 and the Darks as 180.5 and 180.6...?

BPP with CosmeticCorrection & calibrate only -> StarAlignment -> ImageIntegration:

WBPP = No CosmeticCorrection, Subframe Weighting (all three boxes checked and Galaxy selected), Image Registration:

I'm pretty happy with the results of WBPP!!! What say you?

[UlteriorModem]

I'm pretty happy with the results of WBPP!!! What say you?

I will let you know when I produce results. Moving slowly … patience

[Stuart]

Thanks yea I had heard about this. Is the idea it can find and reject 'bad' frames?

Don't quote me on this but I believe it's more of a weighting (think of it as a rating system, sort of) on how "good" of an image is and which are better than others. It then orders them from "good to bad..." It can also select the best reference file to use, based on that weighting.

I can explain.

tl;dr: It brings in the subframe selector script and stacks an image based upon that.

Long explanation
A workflow for preprocessing means that you have to stack your bias and dark frames, calibrate your flat frames, stack your flat frames and calibrate your light frames. You can go further and do cosmetic correction. And it will stack your light frames. That's what the old BPP did for you. For many people that was sufficient. Plugging all your frames, running BPP, getting up and making a sandwich, and finding your master lights all stacked was fine.

The problem with BPP, though, is that you have to manually pick your registration frame--which is actually fine--but it's not necessarily your "best" frame. Also, it stacks using Noise Evaluation which may or may not be the best way to stack images.

What Subframe Selector does for you is that it allows you to do a mathematical measurement to weight your images based upon that expression. It will also calculate and pick your "best" frame depending on the expression. So many of us would do BPP through cosmetic correction and then manually stack using the SFS script.

Now BPP will measure the frames for you, pick the best frame, and stack based upon SFS. You can use the drop down menu to decide the best expression to use based upon your image.

That said, it's REALLY important to blink through ALL your subs (including your calibration subs) to throw out the bad frames before you go into BPP.

Hope that helps. Happy to answer any follow up questions you might have.

Stuart

[JimMinCT]

Ah I see. I have had a chance to try it out. Took a few tries but I got it working. But got an odd error when I went to stack the results!

PI Error by Tom Whit, on Flickr

Thing is though it stacked just fine with good results!

Tom, You should send that screen shot in to Juan on the PI forums. He really does reply almost instantly about these types of things...
I had an issue with the batch file conversion script and showed him the screenshot. Within hrs, there was a bugfix update posted.

I think the weights are used to determine how much of an image is noise and how much is signal, and then to rank them accordingly.
We had a discussion on here a while back about what exactly PI is doing in the background.

Following this thread closely...

[UlteriorModem]

I already have and the issue is more or less resolved. It was stated that this is a long standing 'bug' in the program and is related to site meta data (Lat & Lon) embedded in the .fit frames and should have been corrected in the latest release.

It really has nothing to do with WBPP evidently.

I have to put this on the back burner for a bit as right now I am trying to get the guide camera to work better with the narrow band filter in front of it.

[fatboy1271]

I can explain.

Stuart

Thanks Stu!

[Stuart]

I'm just running it for the first time myself. I'm not seeing a cosmetic correction option on it. I asked Warren about it.

[Stuart]

Oops. Never mind, my bad. I found it.

[fatboy1271]

I'm just running it for the first time myself. I'm not seeing a cosmetic correction option on it. I asked Warren about it.

Oops. Never mind, my bad. I found it.

My laptop isn't online, but isn't in the same place?

[UlteriorModem]

Yea it's there.

Something I don't understand about it.

Okay it weighs the subframes and embeds keywords into the images.

When I run it it just spit out 15 out of 15 frames registered. Its as if no sorting or culling had been done?

Also for it to work is it best to run the image integration from within the script, despite the stern warning to not trust the results?

[Stuart]

Hey UM,
It will spit out all the frames. It doesn't "cull"--i.e. it won't reject images like the regular SFS. You have to blink through them before you run it to throw out the baddies. The weighting comes into play when you stack the image. Normal weighting uses Noise Evaluation as the weighting tool. The SFS in turn uses the SFS weighting instead of noise evaluation.

And yes, you get that warning. That's always been there--I just click the box "Got it, don't show again". I find that the final result out of it is perfectly acceptable as long as I've done my pre-WBPP blink.

And in terms of the cc: I just missed it. I was looking in the wrong box.
Stuart

[UlteriorModem]

Well does it sort the subframes according to its 'score'?

I don't see any indication of the relative weights in the filenames or elsewhere.

[Stuart]

Hey UM,
The weights should be in the FITS data. You have to open the FITS reader to see it. It won't be in the filename.
Stuart

[XCalRocketMan]

Something else to keep in mind. I have been getting memory errors (crashes because protected memory is trying to be read/written) using the new PI and WBPP. I'm not the only one either. So far, it is rare. I was able to rerun the script twice and it finally completed. Common factors so far are Windows 10, NVidia graphics card, but these may be totally unrelated - just happens the systems affected have these in common.