Page 1 of 3

Short exposures + LP = NOISE!

Posted: Mon Oct 07, 2019 2:55 pm
by Larry 1969
Hello all!

I may have reached the limitations of my set up (10" GoTo Dob + DSLR w/ CLS-CCD clip in filter in Bortle 6 skies).
I posted recently in the beginners forum about many short exposures vs fewer long ones and I think I understand now.

Saturday night was exceptional for me! Pretty good skies and very nice tracking! I couldn't get myself to stop! Clouds rolled in at midnight forcing me to call it quits...
I managed 200 20 sec exposures of NGC 7331 hoping to pull out some of the more faint galaxies and with that many exposures I had high hopes.....
After that session finished my target was nearing the meridian so I went to M74 and took another 200. Tracking was still very good.
After that M45 just crested my trees and while I am aware my focal length is too long I still gave it a try. I was able to manage 30 exposures before the clouds ended my session.

I processed all of it (rather quickly) yesterday and I found the images to be quite noisy. I used master darks and master bias frames in the stacks.
I followed Bladekeeper's workflow in StarTools then opened the images in GIMP. I used the "levels" to get the blackest black I could manage without totally ruining the images.

I found M45 requires less processing. I'm not sure if that was due to the brightness of the object or the much fewer subs.

Attached are the images. I will add links to the stacked images if anyone would care to give them a try. I'm curious as to whether the noise can be reduced with better processing.
Here are the links.
Deer Lick group https://www.dropbox.com/s/3f7x1kugh3f2o ... k.fts?dl=0
M74 https://www.dropbox.com/s/ljmosvhw94rt25q/M74.fts?dl=0
M45 https://www.dropbox.com/s/3l3okhjsk4ygfao/M45.fts?dl=0

Thanks!

Larry

Re: Short exposures + LP = NOISE!

Posted: Mon Oct 07, 2019 3:16 pm
by Richard
Thats Great thanks for sharing Larry

Re: Short exposures + LP = NOISE!

Posted: Mon Oct 07, 2019 5:05 pm
by yobbo89
essentially doing short subs requires higher gain which causes more noise when collecting signal and the noise/ signal goes with the gain increased,there's a thing called unity gain, most manufacture list the best iso or gain settings for their camera as well as other specs. the end result is it should take a little more integration over a longer lower iso exposure but your results are going to similar ,

The reason why your images are so noisy isn't totally on being short exposures ,it's because those targets are very dim and being in bortle 6.unfortunately i don't think going longer subs will help , more integration time will help but those targets are on the limits for your skys.

m74 having a very low surface brightness is a bit of a struggle with a surface brightness off 14.3 magnitude of 9.4
m45 also has a low surface brightness of 11.14 but is a lot better in magnitude of 1.2
ngc 7331 has a surface brightness 13.1 and a magnitude of 9.4

I would say m45 is the brightest target to image but it's producing different wavelengths and it's whole area has different surface brightness level spots and it does look like its very close regions of nebula near the stars have less noise.

after i spend 5+ hours on those targets, my images look like yours, so you're not alone!, they are challenging and can be futile.

Re: Short exposures + LP = NOISE!

Posted: Mon Oct 07, 2019 5:29 pm
by Larry 1969
Thanks for that explanation yobbo89!

They were just in my "sweet spot" of the sky where my limited tracking seems to behave very well.
I'm actually pretty to capture anything with my set up. Just wondering if the noise can be reduced with different processing techniques.

Larry

Re: Short exposures + LP = NOISE!

Posted: Mon Oct 07, 2019 6:42 pm
by bobharmony
Don't be too discouraged, Larry. 200x20 is just over an hour of exposure - you have room for improvement here. And some noise in your finished images isn't a bad thing on its own. The eye kind of expects some of it, and will overlook it in an image with interesting information to look at.

I shoot in very LP-infested skies too. I am curious, what ISO are you setting the camera at? Also would you consider an experiment with removing the CLS filter? I tried one a couple of years back and found it reduced the amount of light I collect, and made getting decent color out of the resulting stack. StarTools is very good at restoring natural colors and can handle a lot of LP. I shoot three minutes subs, and they all are a muddy brown color. After stacking in DSS, they are more grayish, but after running through StarTools they can be restored pretty nicely.

I am positive you are not at your limit just yet. You will improve your results by sticking with it! Nice job framing the Pleiades - with a little tweak you can get them all into your FOV nicely.

Bob

Re: Short exposures + LP = NOISE!

Posted: Mon Oct 07, 2019 7:13 pm
by Larry 1969
bobharmony wrote: Mon Oct 07, 2019 6:42 pm I am curious, what ISO are you setting the camera at? Also would you consider an experiment with removing the CLS filter? I tried one a couple of years back and found it reduced the amount of light I collect, and made getting decent color out of the resulting stack. StarTools is very good at restoring natural colors and can handle a lot of LP. I shoot three minutes subs, and they all are a muddy brown color. After stacking in DSS, they are more grayish, but after running through StarTools they can be restored pretty nicely.

I am positive you are not at your limit just yet. You will improve your results by sticking with it! Nice job framing the Pleiades - with a little tweak you can get them all into your FOV nicely.

Bob
Thanks Bob!

I'm very open to experimentation. My camera had been full spectrum modified so I assumed I would need some type of filter. I heard the Astronomik CLS-CCD was a good choice.
I would have no problem removing it and doing a comparison. Of course, nothing but rain here now.....

Also, I'm shooting as ISO 1600. I thought about dropping it to 800 but was afraid of losing data.

Larry

Re: Short exposures + LP = NOISE!

Posted: Mon Oct 07, 2019 7:53 pm
by yobbo89
If your histogram background isn't cliped then you don't "loose data" it almost doesn't realy mater as long as your histogram is between the lower and higher end to fill the well pool as you Can "stretch" the data in processing. All my images are dark and just have the sky background lifted.

Going to 800 iso might reduce noise if your still within your histogram which actual might be better then using 1600,youll have to experiment but make sure your histogram isn't saturated and has the sky background lifted.

Re: Short exposures + LP = NOISE!

Posted: Mon Oct 07, 2019 10:12 pm
by Larry 1969
Thanks again yobbo89!

I'm using APT for capture. It has the histogram split up into LRGB channels instead of a single one like you posted.
I assume all 4 channels are equally important? Sorry but this histogram stuff is uncharted territory for me......

Larry

Re: Short exposures + LP = NOISE!

Posted: Mon Oct 07, 2019 10:27 pm
by Juno16
Hi Larry,

Very nice images!

APT states “The histogram area is divided in 5 logical sectors. For astro imaging is good to inspect the all the channels and to select exposure / ISO combination that places the peaks in the second or/and third sector“. Like yobbo89 said, if you could drop your iso down and still keep the peaks in the second sector, you should hopefully see less noise in your images.

The Levels adjustment is very tempting, but try to use it carefully. It’s easy to make background space darker than it naturally is.

You are pulling in some decent images of dim targets. Fine work!

Thanks,
Jim

Re: Short exposures + LP = NOISE!

Posted: Tue Oct 08, 2019 2:26 am
by bobharmony
Larry, That APT stretched image looks pretty nice, it appears to me you are getting some decent data to work with. The histograms appear to be separating from the left side wall, which is a good thing.

I don't have a modded camera, so take this next comment for what it's worth. Maybe one of our modded buddies can correct me if I am wrong. What is needed for filtering for a modded camera is a filter that blocks the far end of the Infrared spectrum, to keep stars from bloating in your images. The general name is an IR-cut filter. The Astronomic filter you are using has that feature, but it also blocks some wavelengths from the visible spectrum as well, so it is doing double duty, and reducing the amount of usable signal you could be receiving. An IR-cut filter that is specific to the task would let more signal through, which would in turn help with noise.

If you are willing, you could post up one of your stacked images and let someone take a shot at processing it. It looks to me like you have the image acquisition part of AP down pretty well, but post-processing is another beast entirely. Based on the APT screen shot, and your description of how you got rid of the noise from your images, I think there is room for improvement on the data you've got.

There's lots to learn for AP, you are mastering the skills in a sequential fashion. I can say with certainty that after 4 years of collecting photons, I am still not happy with my post-processing skills, but if we start with good data, the rest will come in time.

Bob

Re: Short exposures + LP = NOISE!

Posted: Tue Oct 08, 2019 10:28 am
by Larry 1969
Juno16 wrote: Mon Oct 07, 2019 10:27 pm Hi Larry,

Very nice images!

APT states “The histogram area is divided in 5 logical sectors. For astro imaging is good to inspect the all the channels and to select exposure / ISO combination that places the peaks in the second or/and third sector“. Like yobbo89 said, if you could drop your iso down and still keep the peaks in the second sector, you should hopefully see less noise in your images.

The Levels adjustment is very tempting, but try to use it carefully. It’s easy to make background space darker than it naturally is.

You are pulling in some decent images of dim targets. Fine work!

Thanks,
Jim
Thanks Jim!

So, if I set my ISO to 800 and take a single 20 sec image and my histogram is in the 2 - 3 sector that's good data?

In "Levels" I click on the left "black" eye dropper thing and click on the darkest part of the image. That darkens up the background a bit. Not sure if it's too much or not.

Larry

Re: Short exposures + LP = NOISE!

Posted: Tue Oct 08, 2019 10:37 am
by Larry 1969
bobharmony wrote: Tue Oct 08, 2019 2:26 am Larry, That APT stretched image looks pretty nice, it appears to me you are getting some decent data to work with. The histograms appear to be separating from the left side wall, which is a good thing.

I don't have a modded camera, so take this next comment for what it's worth. Maybe one of our modded buddies can correct me if I am wrong. What is needed for filtering for a modded camera is a filter that blocks the far end of the Infrared spectrum, to keep stars from bloating in your images. The general name is an IR-cut filter. The Astronomic filter you are using has that feature, but it also blocks some wavelengths from the visible spectrum as well, so it is doing double duty, and reducing the amount of usable signal you could be receiving. An IR-cut filter that is specific to the task would let more signal through, which would in turn help with noise.

If you are willing, you could post up one of your stacked images and let someone take a shot at processing it. It looks to me like you have the image acquisition part of AP down pretty well, but post-processing is another beast entirely. Based on the APT screen shot, and your description of how you got rid of the noise from your images, I think there is room for improvement on the data you've got.

There's lots to learn for AP, you are mastering the skills in a sequential fashion. I can say with certainty that after 4 years of collecting photons, I am still not happy with my post-processing skills, but if we start with good data, the rest will come in time.

Bob
Thanks Bob!

That screenshot is not mine. Just one I found online to show the LRGB histogram. BTW, that histogram has all of the peaks in the 1st segment? According to APT that's not right correct?

The filter I'm using is supposed to block I/R and LP. If I were to get an I/R filter would it be a CLS? I see them pretty regularly on CN.

There are links to all 3 stacked images in my original post.

Thanks again!

Larry

Re: Short exposures + LP = NOISE!

Posted: Tue Oct 08, 2019 12:03 pm
by bobharmony
Thanks for the clarification, Larry. Sorry for misinterpreting the APT image.

Whether to use a Light Pollution filter is one of those "religious" areas. Some of us are not fans, some swear by them. Modding the camera doesn't change the cameras response to LP, so the only filter "necessary" for a modded camera blocks the upper ranges of Infrared.

When I get back to my AP laptop, I will download the M45 data and see what I can find in there. It may be a couple of days. If you are OK with it, I will post my results here in the thread, or if you prefer, I will IM you with the information.

I would be interested in seeing what the histogram you get out of DSS looks like (assuming you are using DSS for stacking).

Bob

Re: Short exposures + LP = NOISE!

Posted: Tue Oct 08, 2019 12:05 pm
by Juno16
[/quote]
Thanks Jim!

So, if I set my ISO to 800 and take a single 20 sec image and my histogram is in the 2 - 3 sector that's good data?

In "Levels" I click on the left "black" eye dropper thing and click on the darkest part of the image. That darkens up the background a bit. Not sure if it's too much or not.

Larry
[/quote]

Yes, your exposure should be good if your peaks are in the 2nd or 3rd sector from the left.
Maybe some more experienced users of APT can offer advice on the histogram.

Process your image to what look good to you! I like the background dark too, but levels can reduce faint dso details.
Really a personal preference.

Thanks,
Jim

Re: Short exposures + LP = NOISE!

Posted: Tue Oct 08, 2019 4:50 pm
by Star Dad
Here's an interesting article on ISO settings. Perhaps many of us (me included) are quite wrong about ISO and noise for AP. For your consideration: https://petapixel.com/2017/03/22/find-b ... nge-noise/

Re: Short exposures + LP = NOISE!

Posted: Tue Oct 08, 2019 5:33 pm
by yobbo89
Star Dad wrote: Tue Oct 08, 2019 4:50 pm Here's an interesting article on ISO settings. Perhaps many of us (me included) are quite wrong about ISO and noise for AP. For your consideration: https://petapixel.com/2017/03/22/find-b ... nge-noise/
i'm skeptical about those test from the author and in fact that every dslr camera is different,dose he not contradict his results at the end of the article ? did i read it wrong ?

I would like to see a test of iso 3200 @ 15 sec f2,iso 1600 @ 30 sec f4 and iso 800 @ 60 sec f6 and so on.

Re: Short exposures + LP = NOISE!

Posted: Wed Oct 09, 2019 12:17 am
by Larry 1969
bobharmony wrote: Tue Oct 08, 2019 12:03 pm Thanks for the clarification, Larry. Sorry for misinterpreting the APT image.

Whether to use a Light Pollution filter is one of those "religious" areas. Some of us are not fans, some swear by them. Modding the camera doesn't change the cameras response to LP, so the only filter "necessary" for a modded camera blocks the upper ranges of Infrared.

When I get back to my AP laptop, I will download the M45 data and see what I can find in there. It may be a couple of days. If you are OK with it, I will post my results here in the thread, or if you prefer, I will IM you with the information.

I would be interested in seeing what the histogram you get out of DSS looks like (assuming you are using DSS for stacking).

Bob
So do you think it would be worthwhile to try imaging with no filter at all or should I get an I/R filter?
You can do as you wish with the M45 data or any other data for that matter. It's nowhere good enough to worry about. I appreciate it!
I can post a screenshot of the histogram out of DSS. I always clicked the "link settings" box and drag it to the right a bit to see what I got.

Thanks a bunch Bob!

Larry

Re: Short exposures + LP = NOISE!

Posted: Wed Oct 09, 2019 2:32 am
by bobharmony
Larry - I took a quick pass at your M45 data in StarTools. It isn't an APOD, but it does show that you caught a good amount of the nebulosity there.
M45 Larry.jpg
I had a hard time getting the color right (did I mention that I don't do well with LP filters). I think it is worth a shot to collect some data without the CLS filter in place. If nothing else, it will allow more signal to hit the sensor, which may help a bit with noise, and help StarTools get the color balance better.

I'm guessing that you have some sort of color calibration happening in DSS as well, maybe per-channel background calibration or the like. It may help StarTools to shut as much of that as possible off, to let it get as close to the raw data as possible. SOme of the other settings to turn off are the "set black point" and "use camera white balance". Turning those settings off and restacking might give you better data to take into StarTools.

Not bad for a bunch of 20 second exposures, and I don't think you have hit your limit yet. Your first AP attempts look much better than mine did, that's for sure! Add

Bob

Re: Short exposures + LP = NOISE!

Posted: Wed Oct 09, 2019 10:31 am
by Larry 1969
I will go back and check those settings in DSS.
I notice in StarTools that when I get to the "color" module it turns all of my images very green. I need to pull the green slider up to the blue to get it close.
It's so bad I thought about skipping the color module all together because sometimes I feel it makes the image look worse.

That M45 data is the result of only about 27 stacked frames. (Clouds rolled in)

Thanks again!

Larry

Re: Short exposures + LP = NOISE!

Posted: Wed Oct 09, 2019 1:49 pm
by Juno16
Hi Larry,

I’m not sure of your Startools workflow, but just recently I started using ver 1.5 and now 1.6 alpha.
Previously, I ran Autodev just to find the stacking artifacts, then canceled and cropped. Autodev scare the heck out of me. Then, I Wiped and Developed manually to my liking. I always had green dominance when I started the Color Module and had to really fight with the balance , but never was easy or Quite right.
A few weeks ago I got some help from Ivo with an M31 image. He sent me a workflow. The workflow was pretty much what he outlined in the Startools ver 1.5 Modules Manual available here https://aife.me/startools-prod-pdf/modules
Now I Autodev, Bin, Crop, Wipe, and another Autodev redo global stretch. After that, I run whatever modules I like (usually Decon and HDR) and when I get to Color, the image is pretty close to right. I usually cap green just to get rid of a few greenish stars.
Then Denoise and done.
If you pretty much follow a similar process, maybe it has something to do with the cls filter like Bob mentioned. I don’t use filters.

Thanks,
Jim