Cannot Get Rotated Pictures To Keep Orientation When Uploaded

I'm trying to upload some .jpg pictures to a website, and I'm having a weird issue. There are pictures that are rotated 90 degrees (from the camera being turned when the picture was taken) which I have rotated to the correct alignment in Windows (tried both right clicking and through Windows Photo Viewer), which works great. When I click on the in File Explorer they are exactly how I want them. However once they get uploaded, they revert back to the original orientation which is skewed by 90 degrees. I'm not talking about using them within HTML to build a site, it's a site where you can upload images for viewing. How are the characteristics of an image stored? I can't figure out why they wouldn't hold the alignment.

 

When you make any changes, including rotation, unless you save that edit in the 'original' even though it will show the edit in File Explorer, it isn't permanent. I just did that with a pic as an experiment. If you are saving the edit and it still uploads wrongly, then I have no idea why that's happening.