HTML Logo by World Wide Web Consortium (www.w3.org). Click to learn more about our commitment to accessibility and standards.

Moving forward with Composr

ocPortal has been relaunched as Composr CMS, which is now in beta. ocPortal 9 will be superseded by Composr 10.

Head over to compo.sr for our new site, and to our migration roadmap. Existing ocPortal member accounts have been mirrored.


clean up attachments

Login / Search

 [ Join | More ]
 Add topic 
Posted
Rating:
#73276 (In Topic #15334)
Avatar

Honoured member

Hi,

over one year of using ocPortal for my small community site the attachments disk space has increased to over 200MB. Many files are stored twice, thrice … (several uploads) but are not longer in use. I thought these files would be deleted, but nothing has happened.

An excerpt from the tutorials:
The attachment will remain in the system until all content that using it is edited to stop using it, or deleted.
When I removed the attachment tags inside a post or a comcode page, that shall delete it? But it doesn't work.

So how could I clean up my attachments? The database table "ocp5_attachments" shows about 900 datasets. I can't do the job manually.

Best regards
Falko

ANDROMEDA - Das junge Beobachterforum :)
Back to the top
 
Posted
Rating:
#73279
Avatar

The files should get deleted, but if file permissions are lost, something has confused it, or the WYSIWYG editor has been used, they might not be.

There's no cleanup tool I'm afraid.

Really our philosophy has been that we're in the age of huge hard disks, far bigger than users could fill, so space should not matter. Maybe your web host is cheaping out on their limits?

I'd post a tool to solve it for you but unfortunately I'm not sure it would be easy. I'll run some tests now to see if there is any obvious bug here or something I can do.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#73283
Avatar

Honoured member

Hi Chris,

thanks for quick reply :)

The problem exists since my first installation of ocPortal 5.
At that time I used the quick installer and everything looked good.
After my upgrade from 5 -> 7 I used the fixperms.sh via SSH.
We always used the attachment option to upload the files.

Chris Graham said

Really our philosophy has been that we're in the age of huge hard disks, far bigger than users could fill, so space should not matter. Maybe your web host is cheaping out on their limits?
Yeah, thats right. But it wouldn't be ok, when old (not used) attachment are not deleted. Sometimes you should empty the trash ;)

I'm interested in that tool :)

Best regards
Falko

ANDROMEDA - Das junge Beobachterforum :)
Back to the top
 
Posted
Rating:
#73312
Avatar

Hi,

I just ran 3 tests…
  1. Deleting news item with attachment
  2. Removing attachment when editing news item
  3. Removing attachment when editing Comcode page

In each case the 'physical file' and the database records were correctly erased.

So I think in your case it was probably with the WYSIWYG editor on, because yes I know for sure that will stop attachment reference counting working. This is because it knows attachments may be deformed when working inside the editor, or copying between editors, and so it just locks the attachments in.

Now I know you're a smart guy, so maybe you want to write some kind of tidy-up minimodule script for this. I guess the issue for you is mostly forum posts.
Roughly (this code is untested) something like this would help…

Put in adminzone/pages/minimodules_custom/cleanup_attachments.php and call up with /adminzone/?page=cleanup_attachments

Code

/* Finds attachments that have a reference in a forum post,
      and that post no longer exists
      and there are no other references other than 'null'-type references (used by attachment_safe to hold an attachment in place)
*/
$deleted_post_attachments=$GLOBALS['FORUM_DB']->query('SELECT a.a_url FROM '.$GLOBALS['FORUM_DB']->get_table_prefix().'attachments a JOIN $GLOBALS['FORUM_DB']->attachment_refs r ON r.a_id=a.id WHERE r_referer_type=\'ocf_post\'
   AND NOT EXISTS (SELECT id FROM $GLOBALS['FORUM_DB']->f_posts p WHERE p.id=r.r_referer_id)
   AND NOT EXISTS (SELECT id FROM $GLOBALS['FORUM_DB']->attachment_refs r2 WHERE r2.id<>r.id AND r2.r_referer_type<>\'null\')');
echo 'Run this on linux server command line to delete files...<br />';
echo 'unlink '.implode(' ',array_map('rawurldecode',collapse_1d_complexity('a_url',$deleted_post_attachments)));
/* This could be written as code to actually delete the files, as follows, but better to play safe at first:
foreach (array_map('rawurldecode',collapse_1d_complexity('a_url',$deleted_post_attachments)) as $file)
   unlink(get_file_base().'/'.$file);
*/


or possibly actually it is hanging null references. If so the following should (again, untested) find them…

Code

/* Finds attachments that reference a 'null'-type (used by attachment_safe to hold an attachment in place)
      and there are no other references
*/
$deleted_post_attachments=$GLOBALS['FORUM_DB']->query('SELECT a.a_url FROM '.$GLOBALS['FORUM_DB']->get_table_prefix().'attachments a JOIN $GLOBALS['FORUM_DB']->attachment_refs r ON r.a_id=a.id WHERE r_referer_type=\'null\'
   AND NOT EXISTS (SELECT id FROM $GLOBALS['FORUM_DB']->attachment_refs r2 WHERE r2.id<>r.id)');
echo 'Run this on linux server command line to delete files...<br />';
echo 'unlink '.implode(' ',array_map('rawurldecode',collapse_1d_complexity('a_url',$deleted_post_attachments)));
/* This could be written as code to actually delete the files, as follows, but better to play safe at first:
foreach (array_map('rawurldecode',collapse_1d_complexity('a_url',$deleted_post_attachments)) as $file)
   unlink(get_file_base().'/'.$file);
*/
The problem is here that I'm not confident the code isn't going to get some false-positives, where the WYSIWYG editor might have broken associations for the attachments.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#73338
Avatar

Honoured member

Thanks Chris!

I'm very sure we always uploaded the attachments via this screen:



But I will check it again and inform you about this problem.

I will try to write this tidy-up minimodule script and check out how the attachments work.
That will be a good excercise for me :nerd:

Best regards
Falko

ANDROMEDA - Das junge Beobachterforum :)
Back to the top
 
1 guests and 0 members have just viewed this: None
Control functions:

Quick reply   Contract

Your name:
Your message: