Page 1 of 2

Wide field processing help required

Posted: Mon Sep 09, 2019 8:06 am
by Jockinireland
So, my second night out with StarAdventurer mount, Canon Eos1100d, Kit 18-55 EFS F3.5 lens at about 24mm and F4.6.

Clouds were tormenting me again so 9 X 90 second ISO1600 darks and bias applied (no flats because I screwed them up !)

Stacked in DSS (first image) and Sequator (second) and minimally processed in photoshop - curves, levels some Carbonis actions.

I feel that there is more in the data than I'm getting out (please tell me if you think thats not the case). So, some questions

  • Is there anything I can do in DSS to get a better output - I tried all permutations of White balance, black point etc but this is the best I could get to - its just flat I think
    Can I do anything to reduce the blue in the sequator one (M39 looks wierd to me )....
    .....and pump up the red without making the noise even worse
    Any tricks for doing flats on such a small lens - i used the same as i do on a larger scope (white T LED panel) but it came out with a horrible criss/cross pattern (the weave of the T)
      Any other suggestions on how to make this better.


    Anyway, here y'go, and thanks for any suggestions

    DSS MW 070919 j.jpg
    MW 070919 Sequator r2 j.jpg

    Re: Wide field processing help required

    Posted: Mon Sep 09, 2019 10:22 am
    by yobbo89
    I think you could improve by finding your sweet spot for your iso and exposure time, along with maybe some processing techniques, background extraction, a bit of colour saturation gain to enhanced the colour ectt

    Also some more integration time could help

    What bortle zone are you in?

    Re: Wide field processing help required

    Posted: Mon Sep 09, 2019 10:23 am
    by yobbo89
    Ps i reckon the second image has more detail, but needs a sky background extraction

    Re: Wide field processing help required

    Posted: Mon Sep 09, 2019 10:41 am
    by OzEclipse
    It's a little difficult to say without knowing sky conditions - transparency and light pollution.

    My camera has similar read noise to yours. a few e-'s at ISO1600.

    I'm posting these for comparison. Taken at a dark sky site with good transparency. Single file with single dark subtraction in camera.

    This is straight from the camera, one exposure, in camera noise reduction(dark frame subtraction) no adjustments in lightroom
    It is exposed a tad over 1 stop higher than yours
    ISO1600 80s f2.8-4394.jpg
    In this image, I have wound the exposure slider down 1 stop to make it roughly equivalent to your exposure.
    iso1600 80s f4-4394.jpg
    And here I have optimised the image to my taste in Lightroom only
    ISO1600 80s f2.8-LRboost-4394.jpg
    Do you have a cache where you can put your raw files for download. I'll have a look at them and have a go at processing.

    Joe

    Re: Wide field processing help required

    Posted: Mon Sep 09, 2019 10:54 am
    by Jockinireland
    yobbo89 wrote: Mon Sep 09, 2019 10:22 am I think you could improve by finding your sweet spot for your iso and exposure time, along with maybe some processing techniques, background extraction, a bit of colour saturation gain to enhanced the colour ectt

    Also some more integration time could help

    What bortle zone are you in?
    Hi, the map says bortle 4 but theres a big old streetlamp at the end of my back yard. but there was a high haaze and 50% moon that night.

    I'd planned on a ;oad more subs but the clouds rolled in - West of Ireland - it happens!

    Thanks for your help

    Re: Wide field processing help required

    Posted: Mon Sep 09, 2019 11:14 am
    by Jockinireland
    OzEclipse wrote: Mon Sep 09, 2019 10:41 am It's a little difficult to say without knowing sky conditions - transparency and light pollution.



    Do you have a cache where you can put your raw files for download. I'll have a look at them and have a go at processing.

    Joe
    Hi Joe, thank you so much. Thats a wonderful image and I'd already seen some of your others which are just stunning.

    I'm supposedly Bortle 4 but a big street lamp just outside my boundary. That night there was a bit of a high haze (I'm not sure on how to objectively measure that) and there was a 50% moon so it was never going to be a great night but it was the first tme I've seen the sky in about a month so I just had to try.

    It would be amazing if you could take alook at the data and give me pointers. When you say a "cache where I can put the data" would that be like putting them in google drive or one drive (both of which I have) and sharing the file. If so i'm not sure how to do that but recon I could work it out.

    If I do that should I be putting in the darks and bias' also or just the light.

    Thanks again

    David

    Re: Wide field processing help required

    Posted: Mon Sep 09, 2019 11:35 am
    by Jockinireland
    OzEclipse wrote: Mon Sep 09, 2019 10:41 am

    Do you have a cache where you can put your raw files for download. I'll have a look at them and have a go at processing.

    Joe
    Hi Joe,

    So, I worked it out and have put the files in google drive at the link below. I've only put in 7 lights because 2 had satellite trails. I've also stuck in the darks and bias (in seperate folders) in case they are of use.



    I hope it works!

    Thanks again

    David

    Re: Wide field processing help required

    Posted: Mon Sep 09, 2019 3:59 pm
    by yobbo89
    appreciate the astro data to tinker with, gives me something to do until some of my targets come around to image.

    I had to do 2 background extractions to get that gradient out. which looks to be direct lp from a direction, can be city or local lp ie street light .

    i used kappa sigma in dss, i might go again and use average or median due to the low signal.

    dss report a low sky background and nice sharp focus 'fwhm' which is good

    Re: Wide field processing help required

    Posted: Mon Sep 09, 2019 4:11 pm
    by yobbo89
    can you show us your flats ?

    Re: Wide field processing help required

    Posted: Mon Sep 09, 2019 4:40 pm
    by Jockinireland
    yobbo89 wrote: Mon Sep 09, 2019 4:11 pm can you show us your flats ?
    Hi Yobbo, that already shows a lot more than I could get. Could you explain what you mean by "background extractions" was that in DSS or post processing- (still trying to work this all out) - I use Gradient xTerminator in Photoshop to get rid of my gradients. What are you using post DSS. And what else have you done to get it to that stage.

    Also, where in DSS do you get the "low background" and the FWHM - I focus in Backyard EOS using FWHM so I hoped focus was OK.

    No flats I'm afraid - as I said in the OP I screwed them up (white T and a LED panel ended up giving flats with a pattern which I think is the weave of the T so they were useless. But outside of the criss cross there was not much else in the flats (see below) that I could see so thought it was worth stacking without.
    MILKY WAY_FLAT_Tv1500s_1600iso_f4-5_20190907-23h27m56s538ms.jpg
    Thanks again

    David

    Re: Wide field processing help required

    Posted: Mon Sep 09, 2019 5:21 pm
    by yobbo89
    I had a go with average stacking in dds and it gave me a lot more noise, it was a no go.

    i use automatic backgorund extractor in pix in sight, other programs have the same or similar thing ie photoshop ectt, this should help even out values on the pixels that are affected by light pollution,vignetting ectt. and give you a better starting point to further process

    I did not exactly record my process on that image of yours unfortunately , but i used a methode off ,load up the image- background extraction - a large histogram stretch and clip out a bit of background (noise and lp) - scnr reduction -green - then another automatic background extraction - used the curves with a large saturation adjustment, balanced the rgb by eye, move onto the lum channel curves for brightness, then noise reduction with tvgdenoise , then a small blurring with atrous wavelet transformation - a very small crop for the bad corners -save as jpeg 90%.


    your flats looks problematic, that t shirt doesn't seem to be dispersing light evenly. i can see that the flats are good elsewhere- the darker corners and center, just the t shirt diffraction , i recommend using layers of opaque perspex spaced from each other . make sure the focus is unchanged from your astro session as well.

    in dds where you load your images at the bottom screen, there are columns that list , file,type,score,dx,dy,angle,date/time ectt. along that row if you scroll all the way to the right there should be fwhm,sky background , those two are the most useful to manually analyze the group and toss out bad subs as well as score column too, get rid of a (if you have any) selection of high fwhm subs and high sky background, lower the better and get rid off low scores subs,higher the better for the score column

    Re: Wide field processing help required

    Posted: Mon Sep 09, 2019 6:46 pm
    by Jockinireland
    yobbo89 wrote: Mon Sep 09, 2019 5:21 pm I had a go with average stacking in dds and it gave me a lot more noise, it was a no go.


    your flats looks problematic, that t shirt doesn't seem to be dispersing light evenly. i can see that the flats are good elsewhere- the darker corners and center, just the t shirt diffraction , i recommend using layers of opaque perspex spaced from each other . make sure the focus is unchanged from your astro session as
    Thanks a million. That's all great input. Guess I know what I'm doing tonight then!

    Re: Wide field processing help required

    Posted: Mon Sep 09, 2019 7:28 pm
    by yobbo89
    I was looking at some of my old data from a dslr and a 50 mm lens , the 2 mins from bortle 1 outwayed 2 + hrs of data from bortle 5 , since your sky is bortle 4, i would recommend next time you do some imaging maybe sink in 1 hr or so ,i reckon you will have a much better time processing, ie more forgiving with separating noise and sky background/stretching signal

    Re: Wide field processing help required

    Posted: Mon Sep 09, 2019 7:56 pm
    by Jockinireland
    Yes, I'd intended to get 30 or 40 subs but the cloud rolled on after 9 and I lost 2 to satellites. But that's the plan for next time. At least it looks like PA and focus are ok so, next time the sky clears......

    Re: Wide field processing help required

    Posted: Tue Sep 10, 2019 12:14 am
    by OzEclipse
    Hi Jock,

    Sorry, I passed out early last night... long day at work. Will download your files and have a go tonight.

    cheers

    Joe

    Re: Wide field processing help required

    Posted: Tue Sep 10, 2019 3:21 pm
    by OzEclipse
    This is the best I could manage
    all processed in Photoshop from your raws and your darks
    stacked-1000px.jpg
    I adopted a different approach.

    Clearly, I haven't brought out as much of the red nebulosity as Yobbo but my processing has much less noise.
    Yobbo has done a much better job of getting rid of that gradient.

    My process:-

    - raw lights median statistical stack in Photoshop
    - raw darks median statistical stack in Photoshop
    - darks subtracted from lights
    - multiple adjustment layers applied for colour and luminosity adjustments

    Joe

    Re: Wide field processing help required

    Posted: Tue Sep 10, 2019 3:53 pm
    by Jockinireland
    OzEclipse wrote: Tue Sep 10, 2019 3:21 pm This is the best I could manage
    all processed in Photoshop from your raws and your darks


    I adopted a different approach.

    Clearly, I haven't brought out as much of the red nebulosity as Yobbo but my processing has much less noise.
    Yobbo has done a much better job of getting rid of that gradient.

    My process:-

    - raw lights median statistical stack in Photoshop
    - raw darks median statistical stack in Photoshop
    - darks subtracted from lights
    - multiple adjustment layers applied for colour and luminosity adjustments

    Joe
    Hi Joe,

    Thank you so much for having a go at that. I really appreciate it. Clearly I've a way to go in learning my image processing but it is useful for me to see that there is a lot more in there than I got out so I need to persevere and keep learning.

    I'll go back and see where I can get to after the advice received and if I can get any further I'll throw it back in here.

    Biggest lesson is that i need a LOT more time on the subject - if only the clouds could be made to understand this!

    But again thank you both for your input. Hopefully with time, perseverance, experience and continued help from good souls like yourselves I'll be able to put up something better than these first efforts.

    Rgds

    David.

    Re: Wide field processing help required

    Posted: Tue Sep 10, 2019 10:29 pm
    by OzEclipse
    David,

    I have created a greatly downsized version of the photoshop file with all layers and masks. You can pick it apart and see what I did.
    The zip file is 3MB. It may not attach (not sure what the size limit is) but I will give it a try.
    stacked-900px.psd.zip
    (3.07 MiB) Downloaded 99 times
    To get the file size down I had to flatten the statistically median averaged lights and darks and get rid of the smart objects resulting from that process. They now appear as single layers called lights and darks.

    I only use Photoshop & Lightroom. My background is that I have been using Photoshop since 1991 and teaching workshops since 2001. I have done a whole series of advanced workshops with Les Walkling, a highly regarded expert in the field of digital imaging. My camera has very low read noise and to date, I haven't found a need to use other specialised astro programs. When I hit the wall, I might.

    The thing to remember is that there is no absolute right or wrong in all of this. There is just rendering intent. Sometimes you want to detect something and you push it harder, other times you want a softer approach, a lighter touch. When I saw that Yobbo had pushed it a bit hard, I went in another direction for contrast. But I sometimes push things hard too.

    cheers

    Joe

    Re: Wide field processing help required

    Posted: Wed Sep 11, 2019 8:12 am
    by Jockinireland
    OzEclipse wrote: Tue Sep 10, 2019 10:29 pm David,

    I have created a greatly downsized version of the photoshop file with all layers and masks. You can pick it apart and see what I did.
    The zip file is 3MB. It may not attach (not sure what the size limit is) but I will give it a try.

    stacked-900px.psd.zip

    To get the file size down I had to flatten the statistically median averaged lights and darks and get rid of the smart objects resulting from that process. They now appear as single layers called lights and darks.

    I only use Photoshop & Lightroom. My background is that I have been using Photoshop since 1991 and teaching workshops since 2001. I have done a whole series of advanced workshops with Les Walkling, a highly regarded expert in the field of digital imaging. My camera has very low read noise and to date, I haven't found a need to use other specialised astro programs. When I hit the wall, I might.

    The thing to remember is that there is no absolute right or wrong in all of this. There is just rendering intent. Sometimes you want to detect something and you push it harder, other times you want a softer approach, a lighter touch. When I saw that Yobbo had pushed it a bit hard, I went in another direction for contrast. But I sometimes push things hard too.

    cheers

    Joe
    Fantastic! Just cant thank you enough

    I've opened it quickly at work - will need to wait till I'm home to dig in and it will be brilliant to follow your full work flow.

    But when I open it its just black. The Dark Median Average layer is set to "Normal" I think I need to change that to "subtract" and when I do that it I see pretty much the image as you posted. I'm guessing the "normal" is just an unintended result of flattening the dark median average? I dont think I need to do anything to the Lights median average layer?

    One quickie already - (and apologies if this is obvious but I'm still very green - about 3 months since I first opened photoshop!) I dont get what the layer mask in the second lights median average layer is doing. I see the layer set to screen so I get that. When I take the background layer into a new file, make a new layer from it and change to "screen" it leaves it lighter and less contrasty than yours with the layer mask. Why's that.

    Rgds

    David.

    Re: Wide field processing help required

    Posted: Wed Sep 11, 2019 8:59 pm
    by OzEclipse
    Hi David,
    Fantastic! Just cant thank you enough
    No problem
    I've opened it quickly at work - will need to wait till I'm home to dig in and it will be brilliant to follow your full work flow.

    But when I open it its just black. The Dark Median Average layer is set to "Normal" I think I need to change that to "subtract" and when I do that it I see pretty much the image as you posted. I'm guessing the "normal" is just an unintended result of flattening the dark median average? I dont think I need to do anything to the Lights median average layer?
    Sorry about that, when I first downsized and saved, it was still 500MB because of those smart objects that remained the same size. The flattening of the dark was the last thing and yes I forgot to set the blending to "subtract" or "difference." I used "difference" blending mode in the original but in this case they do the same thing.


    One quickie already - (and apologies if this is obvious but I'm still very green - about 3 months since I first opened photoshop!) I dont get what the layer mask in the second lights median average layer is doing. I see the layer set to screen so I get that. When I take the background layer into a new file, make a new layer from it and change to "screen" it leaves it lighter and less contrasty than yours with the layer mask. Why's that.
    The duplicate layer set to screen adds pixel from that layer to the layer below. Effectively an intensity build that you could also perform with curves or levels. But the alpha mask made from the image only applies the build to the brighter parts of the image without increasing background. You can disable/enable the mask by right clicking on the mask to contrast the effect with/without the mask.

    Regards
    Joe