HTML Logo by World Wide Web Consortium (www.w3.org). Click to learn more about our commitment to accessibility and standards.

Moving forward with Composr

ocPortal has been relaunched as Composr CMS, which is now in beta. ocPortal 9 will be superseded by Composr 10.

Head over to compo.sr for our new site, and to our migration roadmap. Existing ocPortal member accounts have been mirrored.


Provide the ability for member to dynamically filter content

Login / Search

 [ Join | More ]
 Add topic 
Posted
Rating:
#76660 (In Topic #15804)
Avatar

Community saint

Not sure if this has been brought up before or not but I can't find it if it has.

ocPortal needs the ability for the user to dynamically filter content for a minimum of Galleries and Catalogs. The user should see all the items in a selected node and then be able to filter and then further refine the search. This is a pretty common function to a lot of software today and I see it as a glaring missing hole.

I am not sure whether the recent changes to the catalog system might make this doable or maybe you could just use keywords (which would work across multiple content types.

Has any consideration been given to adding this feature?

Bob
Back to the top
 
Posted
Rating:
#76664
Avatar

This is a pretty common function to a lot of software today and I see it as a glaring missing hole.

The problem is, for the software it is common in, it is usually custom coded for a specific scenario. i.e. the interface and search code are not general things, they are hard-coded to specific data sets, be that property, auction items, or whatever the site does.

This kind of stuff does not generalise all that well. The interfaces generally get custom designed to meet the search needs of the end user. That might mean somethings are 'ORd', some things are 'ANDd, some things are strict searches, some things are substring searches, some things are 'greater thans', some things are natural language searches, and so on.

Also search performance is something very hard to give even with custom code – how the query is designed, and how the database is indexed and structured, they usually get a lot of very careful thought based on exactly what is being stored. Whole books are written on query optimisation, and require pre-knowledge of things such as what is the value dispersion in a particular field, or what order to do the search clauses in to get best index usage.

Of course, for most people, performance won't be a big deal. It only becomes an issue after about 10,000 records. But the UI stuff definitely is in my view.

I think all can be solved, but it's probably a hard weeks work.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#76666
Avatar

Currently a simple filter system can be done using the search module. The search block can also be customised (not easy but it can) to offer any kind of searching that the full search form allows.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#76669
Avatar

Community saint

Geez, Chris, you had me pretty dismayed until the last paragraph.

Do you think you will be able to deliver something like this in a generalized manner such that it becomes part of the core code? I am always most interested in seeing that features I want are built in to the core code at a later date.

I've got my eye on one tracker item for next year and am thinking this might also be on my list, especially if we can get some people to split the cost of this one.

Bob
Back to the top
 
Posted
Rating:
#76674
Avatar

Yes, I think it could be generalised. Some kind of scheme for defining exactly what fields work as filters, and what mode(s) the filters work in (i.e. substring, natural search), and then somehow construct the templates in a way that allows the layout of the filter form to be finely tuned. Probably everything would be considered an 'AND' clause.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#76679
Avatar

Community saint

I'm going to collect my thoughts on this and then put it in the tracker. I know it's going to be expensive but I think it would be a huge improvement for ocPortal.

I know I could use it right now.

Bob
Back to the top
 
Posted
Rating:
#76699
Avatar

You're actually very lucky, because right now I am implementing 50% of this for a client. I think my mind wasn't quite in gear yesterday (think I had a cold) because on some level I must have known that it was at the top of my to do list, lol.

It's not going to take it to the next version on its own, but if someone does sponsor it I know all the backend code can be grabbed from this project and I'd just need to focus on things like block parameters and template interfaces.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#76700
Avatar

Community saint

Chris-

That's super to hear. I will write up a feature request for the tracker and we will go from there. I won't be able to do anything till next year (well, unless I hit the lottery big time in which case you will have a sponsor for v8) but we can at least get it on the list.

Bob

EDIT: Spelling


Last edit: by BobS
Back to the top
 
Posted
Rating:
#76726
Avatar

Community saint

I have submitted the feature request in the tracker:
0000309: Allow dynamic user filtering of catalog and gallery display

Please add any comments or consideration you have so Chris can provide an estimate of what the cost will be.

Also, don't forget to vote for this item if it is important to you Tracker.

Bob


Back to the top
 
Posted
Rating:
#76727
Avatar

Community saint

Chris-

If a person monitors a tracker item in Mantis, does the system automatically cast a vote for the item in your front end? If not, there should be a simple way to get to that item to cast a vote.

Bob
Back to the top
 
Posted
Rating:
#76731
Avatar

BobS said

Chris-

If a person monitors a tracker item in Mantis, does the system automatically cast a vote for the item in your front end? If not, there should be a simple way to get to that item to cast a vote.

Bob

This is how it works, yes.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#76732
Avatar

I don't have time to spec this out at the moment to do a proper cost, but it'll be around a few days.

For the project I mentioned, we have an arbitrary filtering system setup deep inside the catalogues system, and this allows filters to match with the following match types:
  • >
  • >=
  • <
  • <=
  • direct string match
  • substring/wildcard match
  • fulltext search match
Any particular filter can be defined to check against a list of fields ('OR'), and together the filters and 'AND'd.

We wrote the main_google_map and main_cc_embed blocks to recognise these filters, and altered the catalogue templates to display entries via main_cc_embed instead of natively. That works well as it provides much more flexibility.

What we don't have is any kind of interface for showing the filters in a form. Currently the HTML has to be hand-coded and then mapped from parameters sent to the page to parameters to blocks using blobs of Tempcode. So we probably want a block that builds out filter forms, and we probably want main_cc_embed to take a parameter to recognise filters passed to the page via a standard naming convention, and probably we also want the normal catalogues module to do the same automatically.

All this also would be wanted for galleries, which I haven't considered, but should be simpler – except there is currently no provision for searching custom fields, so that might want to be done also.

Generally I was thinking that there be a sensible division of roles on what the search module is for and what module/block filters are for. The search module being a generic thing for finding stuff across the whole site that matches keywords, and filters being for much more structured searching.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#76735
Avatar

Community saint

Chris-

Yes - structured searching is exactly what I would like to see. It seems you have a solid handle on this.

It's to bad that the galleries are not built atop the catalogs which would likely eliminate the need for redundant code. Maybe something for you to consider in v8.

Bob
Back to the top
 
Posted
Rating:
#79829
Avatar

Well-settled

Fantastic idea Chris - will help to sponsor this idea!

Dave
Back to the top
 
Posted
Rating:
#79837
Avatar

Community saint

ntesla27-

Thanks for providing the other half of the support credits or this change. I think this is going to be an excellent new feature and hope that many community members find use for it.

Bob
Back to the top
 
Posted
Rating:
#84542
Avatar

ocPortal Tutorial: Filtering using ocSelect syntax

Written by Chris Graham, ocProducts
ocPortal contains a filtering system for selecting content based upon defined filters. ocSelect is the language for defining filter strings, and is an advanced feature of ocPortal. It is not to be confused with ocFilter, which is the language for determining what categories to return results from.


An introductory example

An ocSelect filter string consists of comparisons, separated by commas. Here is an example:

Code

something=value,somethingelse>value
This defines two comparisons. Both must hold true for any result returned (i.e. they are ANDd).

In this example, the first is an '=' comparison, and the second is an '>' comparison. Anyone who knows very basic maths will understand what these mean.

'something' and 'somethingelse' would be database field names for the main database table of whatever content type is being filtered.

The full language

Security note

For security reasons, some fields are not available to ocSelect, such as password fields
The field names must match field names in the content type's main database table, or one of the following special values:
  • compound_rating (the rating)
  • meta_keywords (the SEO keywords / tagging)
  • meta_description (the SEO description)
You are allowed to specify that comparisons may work on multiple fields, by separating field names with pipe (|) symbols. This is particular useful for a search filter, to make the typed string look across multiple fields.

GET/POST?

GET/POST parameters are the technical name for how parameters are passed to web pages. The &something=value stuff you see in URLs are GET parameters, and POST parameters are what forms pass around behind-the-scenes.
We support the following comparison types:
  • < (Less than)
  • > (Greater than)
  • <= (Less than or equal to)
  • >= (Greater than or equal to)
  • = (Equal to, where blank means skip)
  • == (Equal to, where blank means only a blank value will be matched)
  • ~= (Contains substring)
  • ~ (Seems to match, according to database full-text search matching)

The comparison value can either by given directly, like in the introductory example, or it can be written like <name>, which will tell it to look within a GET/POST parameter called filter_<name>. This is the mechanism by which you hook up forms to the filter strings.

You can also specify that comparison types are read via GET/POST parameter in the same way, if you want users to define how each field should match.

Where the filters can be used

There is an ocSelect-based parameter for the following blocks:
  • main_multi_content
  • main_gallery_embed
  • main_cc_embed

As well as active support within the following modules (TODO: Not implemented yet, reserve right for it to only be for catalogues/galleries):
  • catalogues
  • downloads
  • galleries
  • groups
  • members
  • news
(the filter used will be read from any GET/POST active_filter parameter that is set)

As explained above, ocSelect works by reading GET/POST parameters and matching them using the rules specified in the defined ocSelect filter string. In the case of the filtering for the modules, the filter string is also supplied as a GET/POST parameter.

Generating filters

Thumbnail: Inspecting the block's HTML to get the auto-generated ocSelect filter string, which we'll use as a starting point

Inspecting the block's HTML to get the auto-generated ocSelect filter string, which we'll use as a starting point

Thumbnail: An automatic news filter form

An automatic news filter form

The concept of writing ocSelect filter strings, and matching forms, can be a bit daunting. This is why the main_content_filtering block exists. It will automatically construct filters and forms for content types.

You don't actually need to use this block in production at all. It is a good way to build up a default form, which you can then get HTML from as a starting point.

There are two steps in getting your HTML ready:
  1. Show the block for your content type, with no parameters
  2. Grab the auto-generated ocSelect filter string and customise it, then show the block using your customised filter


Once you have built a default form for a content type, inspect the HTML to get the auto-generated active_filter value (see the image above).

In our case the auto-generated ocSelect filter string was:

Code

allow_comments<allow_comments_op><allow_comments>,allow_rating<allow_rating_op><allow_rating>,allow_trackbacks<allow_trackbacks_op><allow_trackbacks>,author<author_op><author>,date_and_time<date_and_time_op><date_and_time>,edit_date<edit_date_op><edit_date>,news<news_op><news>,news_article<news_article_op><news_article>,news_views<news_views_op><news_views>,notes<notes_op><notes>,submitter<submitter_op><submitter>,title<title_op><title>,validated<validated_op><validated>,compound_rating<compound_rating_op><compound_rating>,meta_keywords<meta_keywords_op><meta_keywords>,meta_description<meta_description_op><meta_description>

We will take that and modify it. In this case I think the following filter is much better:

Code

author~=<author>,date_and_time><date_and_time>,news|news_article|title|meta_keywords|meta_description~=<news>,submitter=<submitter>,compound_rating>=<compound_rating>

Thumbnail: A refined filter form

A refined filter form

We then feed into the block's "param" setting to get our refined filter form (see the image below).

Putting the filter form into the site

If we are happy with how our refined filter form looks, we can continue with the main_content_filtering block.
Otherwise, we can grab the HTML that the block output and tune it. The only thing that ocPortal needs is the GET/POST fields to match what the ocSelect filter string wants, otherwise you can use whatever HTML you want.

We'll assume we'll continue to use the main_content_filtering block for the following continuations of our news filtering example.

Putting into a panel

If you want to place a filter on the news page, with it shown in a panel only on the news page, you'd put something like this into your panel's Comcode:

Code

{+START,IF,{$MATCH_KEY_MATCH,site:news}}[block="author~=<author>,date_and_time><date_and_time>,news|news_article|title|meta_keywords|meta_description~=<news>,submitter=<submitter>,compound_rating>=<compound_rating>" content_type="news" labels="date_and_time=Newer than,news=Containing,submitter=Submitted by,compound_rating=Minimum rating" types="date_and_time=days"]main_content_filtering[/block]{+END}

Putting into a news template

If you wanted it on the news archive screen, you'd edit the NEWS_ARCHIVE_SCREEN template and add:

Code

{$BLOCK,block=main_content_filtering,param=author~=<author>\,date_and_time><date_and_time>\,news|news_article|title|meta_keywords|meta_description~=<news>\,submitter=<submitter>\,compound_rating>=<compound_rating>,content_type=news,labels=date_and_time=Newer than\,news=Containing\,submitter=Submitted by\,compound_rating=Minimum rating,types=date_and_time=days}

Concepts

ocSelect
ocPortal's syntax for filtering returned content

See also



Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#84543
Avatar

(posted the wrong tutorial first time around, posted the right one now)

This is 80% done now. I just haven't hooked the filters in everywhere yet, or added options to allow subtree result collation (i.e. you can get results for subcategories merged in).


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#84544
Avatar

Forgot to mention, but custom fields are supported too.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
Posted
Rating:
#84545
Avatar

Community saint

This seems very thorough but I am a bit concerned how this will be presented to the site visitors who will actually be using it. I had expected some sort of tree that allows adding new search values from a preset collection to narrow the search much like the "refine search" on a number of shopping sites.

Bob
Back to the top
 
Posted
Rating:
#84546
Avatar

The list field type will do that. It'll look at all existing field values and allow the user to select which one they want. I will think about allowing multiple selection lists, and also link-based filtering (i.e. you click successive links). That might be what you were thinking, but essentially we are very close to that already – it's just a slightly different interface. Drill-down works because you don't need to fill in every filter on the form.


Become a fan of ocPortal on Facebook or add me as a friend. Add me on on Twitter.
Was I helpful?
  • If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
  • If so, please let others know about ocPortal whenever you see the opportunity.
  • If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying ocPortal on fun personal projects.
  • If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
Back to the top
 
There are too many online users to list.
Control functions:

Quick reply   Expand