HTML Logo by World Wide Web Consortium (www.w3.org). Click to learn more about our commitment to accessibility and standards.

Moving forward with Composr

ocPortal has been relaunched as Composr CMS, which is now in beta. ocPortal 9 will be superseded by Composr 10.

Head over to compo.sr for our new site, and to our migration roadmap. Existing ocPortal member accounts have been mirrored.


[SOLVED!] Corrupted Downloads!

Login / Search

 [ Join | More ]
 Add topic 
Posted
Rating:
#75588 (In Topic #15644)
Avatar

Community saint

I upload a file to site, but the upload gets corrupted

Greetings,

Ok, here's the situation. As you may or may not know, I run a forum based RPG that is based on my table top RPG. This RPG uses a stat system, and uses the ocPortal Point System so that members can use points to gain stats and what not.

For a little while now I've been using the catalogue system to build character profiles. I made a three part profile, and members filled it in to make their character. In order to try and simplify things some, I made some of the fields drop-down lists. It works ok… but as we are still experimenting with certain aspects of the system to make the RPG work better, there are occasionaly stat, armor, or so on system changes/fixes.

We just initiated a large change to how armor works, which would basically require the participants to redo their armor stats and such, not always the funnest thing to do.

Since I cannot implement spreadsheet-like functions (auto-calculation and the like) into the catalogue based profile, I figured I'd make things a little easier on everyone, and just make a spreadsheet that is cross-platform capable. So, firing up my trusty ole' Libre Office Calc, I make the character profile spreadsheet, full of auto calculation goodness that would make creating and managing characters a walk in the park, and make it easy to assign future system fixes, and for staff to help with profile management. As most people in this world still rely on Microsoft Office, I saved the file in a compatible xls format. I then, to lead by example, made my primary character profile in it.

I uploaded the profile form as a new download, and I attached my character's profile to my signature, showing again how easily this would all work.

Well that, my friends, is when the trouble started…

I announced the idea, that we'd test it along side the catalogue version, and let the members weigh in on which they liked better. Several immediatly jumped in and sent me messages about how it only opened up as a load of gibberish. Well opening it on my computer was fine for obvious reasons, so I went to my wife's laptop, downloaded, and tried to open in both MS Excel 2010 and Libre Office Calc. Gibberish…

Flustered, I preceeded to save in multiple formats on my desktop, and test in both programs on my desktop, upload them as seperate downloads, and even tried to upload a zip file containing all three (which somehow changed to a different file entirely in the downloads). All files, when downloaded on my wife's laptop… gibberish or woudln't even bother opening. The zip file that the site thought needed to be named differently… it was empty!

Ok, so… now what? I decide it's time to play the fun game of "Process of Elimination". First, is it the files themselves? I take the files, via my handy little flash drive, straight from my pc to my wife's laptop. They work just fine. So the files are ok.

Is it my internet? I upload the files to dropbox, download to my wife's laptop (making sure that I deleted the others to prevent false positives). They work fine. Internet demons not to be blamed this time.

That leaves two last possible culprits, ocPortal or opensource hosting (my hosting provider, or the server I'm on). I am hoping it is a simple ocPortal bug that Chris can march in here two seconds before I post this with a cure as he's so awesome at doing. Or maybe some obscure ocPortal function I may have overlooked that others who use downloads more can direct me to. If it's the server, I'd be screwed until Feb. when I can think about looking for another host if need be.

So please, any ideas? Any fixes?

Thanks, in advance for reading my long ramble and any solutions. 


Last edit: by mythus

Legends of Nor'Ova: A site powered by ocPortal; home of the Legends of Nor'Ova tabletop RPG wiki and community.

Like ocPortal? Want to thank Chris and gang somehow? Then help out in the chat room! It really needs your help! Just open it in a tab everytime you open your web browser, and when you hear a "ding", check it out!

"Those who want help should first be willing to give help."
Back to the top
 
Posted
Rating:
#75591
Avatar

Different file size? Smaller or larger? Consistently the same wrong size or randomly different?

Could you confirm, this is just catalogue file fields? or the downloads system? or both?


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#75592
Avatar

Community saint

A couple of other tests I can think of:

* Upload from your wife's laptop to your site.

* Upload from your wife's laptop to your site via ftp (assuming you are not already doing so). If it is a site problem then that should determine if the problem is ocPortal or your host.

I've deliberately kept your computers involvement to a minimal to further isolate it as a potential cause.

I had a very nasty problem many years ago where my zip files, and therefore my backups, were being corrupted (and of course I didn't know about it until I tried to retrieve something from the archive). Everything else on my pc seemed to be working fine. Long story short, turns out that my RAM had gone wonky in such a way that it only affected (or was noticeable through) largish zip files getting corrupted because of how the archive program was allocating memory.




Do you have a Samsung Galaxy S / Galaxy S II ? If so, why not check out my ScreenFree FM Radio .
Back to the top
 
Posted
Rating:
#75594
Avatar

Community saint

Greetings,

I have tested uploading (just now) from my wife's laptop, with a working file, but it downloaded a corrupted file when downloading the same file from the site.

I will test uploading straight to the sever via ftp, however that would not help with members being able to attach the file to their signature or whatever. But it would at least rule out the host yes?

I do note that the file name and extension gets changed by ocportal in the server. Despite the filetype that is uploaded, it is stored as a dat filetype, though that may be standard procedure.

As for file sizes, a 676.6 KiB file becomes a 677 KB file, but that seems so insignificant and may be tied to the differences between KB and KiB.

And Chris, this is for files attached and uploaded to the download section (and later downloaded).Filetypes affected thus far are zip, xls, xlsx, and odt, all which are selected by default permissible formats.

Legends of Nor'Ova: A site powered by ocPortal; home of the Legends of Nor'Ova tabletop RPG wiki and community.

Like ocPortal? Want to thank Chris and gang somehow? Then help out in the chat room! It really needs your help! Just open it in a tab everytime you open your web browser, and when you hear a "ding", check it out!

"Those who want help should first be willing to give help."
Back to the top
 
Posted
Rating:
#75595
Avatar

Renaming to .dat is for security, to stop them being runnable on server and also to stop people bypassing cat perms by guessing URLs. If you download and rename the dat file, does it work?

(will tell us if issue is in uploading or downloading)


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#75597
Avatar

Community saint

Greetings,

Uploading and downloading via FTP straight to server (desktop and laptop both involved) produced a working file.

I'll check the dat thing for you now Chris.

Legends of Nor'Ova: A site powered by ocPortal; home of the Legends of Nor'Ova tabletop RPG wiki and community.

Like ocPortal? Want to thank Chris and gang somehow? Then help out in the chat room! It really needs your help! Just open it in a tab everytime you open your web browser, and when you hear a "ding", check it out!

"Those who want help should first be willing to give help."
Back to the top
 
Posted
Rating:
#75598
Avatar

Community saint

Renaming the dat file did not help the file. It remained unusable.

Legends of Nor'Ova: A site powered by ocPortal; home of the Legends of Nor'Ova tabletop RPG wiki and community.

Like ocPortal? Want to thank Chris and gang somehow? Then help out in the chat room! It really needs your help! Just open it in a tab everytime you open your web browser, and when you hear a "ding", check it out!

"Those who want help should first be willing to give help."
Back to the top
 
Posted
Rating:
#75607
Avatar

So we think it uploads okay but the downloading breaks it?

Could you send me the original file, and what downloaded please.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#75611
Avatar

Community saint

Sure, I can email them to you. Would you like all the files I have tried? Or just one as an example? And if just one, which file format; the .xls, the .odt, the .xlsx, ,the.zip? Just want to make sure I send you what would best help track this thing down.

I do note that the zip file when I upload it, gets its name changed from SoA Character Profile.zip to LofNRPGBook.zip, which is the name of the zip file that had already been uploaded some time ago for my premium membership group. Since that downloads an empty zip file, makes me now curious, did it empty my original LofNRPGBook.zip file….

Legends of Nor'Ova: A site powered by ocPortal; home of the Legends of Nor'Ova tabletop RPG wiki and community.

Like ocPortal? Want to thank Chris and gang somehow? Then help out in the chat room! It really needs your help! Just open it in a tab everytime you open your web browser, and when you hear a "ding", check it out!

"Those who want help should first be willing to give help."
Back to the top
 
Posted
Rating:
#75615
Avatar

One example. I'll do some kind of hex comparison.

Also FTP access would help.

I do note that the zip file when I upload it, gets its name changed from SoA Character Profile.zip to LofNRPGBook.zip, which is the name of the zip file that had already been uploaded some time ago for my premium membership group. Since that downloads an empty zip file, makes me now curious, did it empty my original LofNRPGBook.zip file….
Probably the "original filename" field on the form was left as it was.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#75618
Avatar

Community saint

Greetings,

Email sent with needed credentials and example file.

Thanks again!

Legends of Nor'Ova: A site powered by ocPortal; home of the Legends of Nor'Ova tabletop RPG wiki and community.

Like ocPortal? Want to thank Chris and gang somehow? Then help out in the chat room! It really needs your help! Just open it in a tab everytime you open your web browser, and when you hear a "ding", check it out!

"Those who want help should first be willing to give help."
Back to the top
 
Posted
Rating:
#75656
Avatar

Community saint

Greetings!

Through email conversation, Chris has solved this issue! The downloads function works fine, no more corrupted files!

As requested, I am sharing the email posts displaying where he fixed it.

Email 1: This is where he almost fixed it

Chris said

Fixed. Was apparently a bug in your PHP version.

FYI the fix was to remove
   header('Content-Transfer-Encoding: binary');
from sources/downloads2.php

because in your PHP it triggered throwing away of the HTTP headers that say the response was gzipped (Content-Encoding), and you just get gzip binary bytes out with the browser not knowing what it is.

Btw, the file you have up is not the xls you sent, it's an exported htm from Open Office.

Could you post what I said here in the topic please.

Chris

My response to Email 1

mythus said

Greetings,

Thanks so much for looking into this for me! I'll gladly make a post with this information. Let me just go and give it a test so I can also include in said post that it works.

Figures it'd still be something related to the server, such as a php issue.. ah oh well, I planned to pay Brian for hosting (assuming his hosting service is still going good then and it doesn't include that damnable php-safe-mode crud) come January/February anyways. At least I had a good year of free hosting to test out this host to see if it would be viable for the long term.

I know one of the files was the xls version, or at one time anyways. I've tested so many formats and whatnot trying to figure this out before beseeching your services it wasn't funny.

So, thanks again Chris!

But wait! We forgot about the sequel! Every good movie has a sequel! So, presenting…

Email 2: Return of the Corruption

mythus said

Greetings,

Just a quick question that hopefully doesn't translate into a large headache, is there anything I am supposed to do on my end? I noticed that you gave me your sollution, am I to apply that to my /sources/ downloads2.php?

Reason I asked is because my test failed…

I uploaded a completed character profile (that I tested beforehand to make sure opened right on both desktop and laptop to prevent false positives), and well I got a corrupted file in return. I am attaching both, just in case.

Notice the discrepancy in file size? The original being a 670 K while the other being 587K? That later one (with the c) is the corrupted one.

Test preformed approximately 1 hour after receiving your email.

But alas, no bug is too mighty, no evil corruption issue too powerful for ocChris, web programming god

Chris said

Whoops. I must have slipped up in my analysis, it was actually a slightly different thing causing it. Fixed now.

I think actually the 'Content-Length' HTTP header was making PHP drop the 'Content-Encoding' header, but keep the content encoding.

And yes, my personal testing confirmed that this is now fixed!

Final Thoughts

This example proves once again the worth and value of ocPortal, that being its most dedicated developer, Chris. While other CMS'es may tempt with flashy looks, catchy catch-phrases, and supposed user friendliness or powerful customization options, none can hold a candle to ocPortal… and that is largely due to it's most powerful and prominent feature, a developer who actually cares about the end user and the success of the product.

Will Chris answer every issue and fix or meet ever need? No, and he shouldn't. He rightly so will direct more complex customization requests and the like to ocProducts, charging a modest fee for them. However, unlike any developer I have ever come across, he personally responds to bug issues and problems with using the core product, and will personally look into them with a fix… even if the bug issue, as in this case, was because of deficient php provided by the host.

I do hope that no one would take such dedication for granted, and do what they can to pay it back. Reaching out and helping one another as Chris does is one good way of doing this, as not only does it make this community wonderful, it also takes some burden off of him. I am delighted to see that this is actually happening here, so many of us taking Chris's outstanding example and applying it ourselves. Notable members doing this being temp, Bob, Fletch, Friendly, and Jean. There are others as well, but as I such at name memory, I mentioned the ones that I remember the easiest.

I also hope that as webmasters we are continuing to follow this example with our own websites, providing bend-over-backwards support to our visitors and members. We should always remember that without them, we do not have a web site.

Ok, I'll close up this long post now. Thanks again Chris. You are awesome, as always.

Legends of Nor'Ova: A site powered by ocPortal; home of the Legends of Nor'Ova tabletop RPG wiki and community.

Like ocPortal? Want to thank Chris and gang somehow? Then help out in the chat room! It really needs your help! Just open it in a tab everytime you open your web browser, and when you hear a "ding", check it out!

"Those who want help should first be willing to give help."
Back to the top
 
Posted
Rating:
#75659
Avatar

Community saint

wot

Take my advice. I'm not using it!

View my working ocPortal site (version 9.x.x) at Anglo-Indian Portal
Back to the top
 
Posted
Rating:
#75675
Avatar

Community saint

Right on, mythus!!

I think I said something similar in a post: ocPortal gas developers who give a damn about the final product and I hope to see that they start enjoying some of the fruits of their tremendous and skillful efforts.

And thanks for one again making the point that we should all make an effort to pay forward the benefits that we enjoy as the result of Chris & Co.

Everyday that passes makes it clearer to me that I made the right choice of CMS.

Bob
Back to the top
 
There are too many online users to list.
Control functions:

Quick reply   Contract

Your name:
Your message: