ocPortal Tutorial: eCommerce
Written by Chris Graham, ocProductsUse the software's eCommerce system to sell things through your website. Our eCommerce system has a wide scope, covering:
- Usergroup subscription (i.e. sell site access)
- Member invoicing
- Online store / shopping cart
- Programmers may also add new product types, including things with digital delivery
eCommerce support is tied to a payment gateway, such as PayPal.
The system is highly extensible, where new dynamically handled and integrated products or services can be added by minimal coding by a programmer.
Table of contents
- ocPortal Tutorial: eCommerce
The eCommerce layering in ocPortal for usergroup subscriptions
The banking system runs with international standards for things such as:
- credit cards
- debit cards
- cheques ('checks' in American English)
Cheque processing is relatively straight forward, but more infrastructure is needed to process credit or debit cards. This is accomplished by using a payment gateway in one of two ways:
- by getting an agent to fully handle the transaction and relay the funds
- by having a merchant account with a bank, using the banks infrastructure and your bank account to manage the transaction yourself
It is the webmasters responsibility to research any payment system before use, to convince themselves they are happy with the quality and terms of service. Whilst ocPortal by default only supports PayPal, WorldPay and SecPay, this should not be considered an endorsement of any of those services.
PayPal was chosen because it is without doubt the leading form of payment on the Internet, easy to setup, and flexible in that it can provide support for funding direct from bank accounts, or cheque, as well as debit and credit card processing, with no need for a merchant account.
WorldPay and SecPay are popular alternatives to PayPal. There are many other alternatives that ocPortal doesn't currently support, but could.
ConfigurationTo configure the payment gateway settings go to "eCommerce options" under the Configuration icon in the Setup section of the Admin Zone.
Config optionsThe options have the following requirement for PayPal:
|Use local payment||This must be disabled.|
|Gateway password||This should be blank.|
|Gateway VPN username||This should be blank.|
|Gateway VPN password||This should be blank.|
|Callback password||This should be blank.|
|Postal address||This should be blank.|
|Email address||For PayPal it is your PayPal email address, but for other gateways it probably is just be any you receive email on.|
|Phone number||This should be blank.|
|[Testing mode] Gateway username||Your PayPal email address (for the testing mode one, it's your sandbox PayPal email address).|
Documentation for other Payment gateways is provided in notes at the top of the sources/hooks/systems/ecommerce_via/<gateway>.php file. Payment systems tend to be a bit complex and ones other than PayPal are best set up by programmers who know the particular ways messages and security works between your site and the gateway.
Test modeAll supported payment gateways provide a test mode, which is turned on from the Admin Zone Configuration. Test mode allows you to make purchases from your website using the full eCommerce architecture of ocPortal and the payment gateway, except no money will actually be charged (it's a simulation mode) – the ideal way to test your set-up without risking real money.
To prevent purchases being made by real people that you don't receive real money for, only members in a usergroup with the 'Access eCommerce system when in test mode' permission may purchase when test mode is enabled ("Purchasing is temporarily offline for maintenance. It should be back online within 24 hours: please try again later." will be seen by others). You have two supported testing choices:
- enable this permission for all usergroups and test the eCommerce when the website is closed (using the SU feature)
- test the eCommerce from your own admin account
For PayPal test mode goes through the PayPal sandbox. Sign up on the sandbox (this is your test account for making the sale), log in, and create a test user (this is your test account for purchasing).
Be advised that we have found the PayPal sandbox to be buggy. If your tests fail with errors like "The link you have used to enter the PayPal system is invalid. Please review the link and try again." then you may be experiencing bugs in the PayPal sandbox. By all means, please complain to PayPal about it – we have run tests and found the sandbox fails midway through purchases on configurations that work correctly for live payments.
ProductsocPortal has an abstract system of 'products' that may be 'purchased'. These ocPortal products are not the same as products that are available from shopping-cart systems, but rather, special types of purchasable product or service that have their own code to handle their purchase.
In ocPortal by default there are the following 'kinds' of 'product':
- usergroup subscription: you may configure as many usergroup subscriptions as you wish, and these are all handled by the 'Usergroup' product kind. Each configurured subscription is considered a 'product' for sale on your site.
- catalogue items: these are items for sale from the online store
- orders: these products are a collection of other products bound into a group, and generated when someone finalises an order from the shopping cart
- work: this is a generic kind of product used to represent something that has been invoiced
- Various kinds of non-purchasable product that exist for accounting purpose – 'interest', 'tax', 'wage' and 'other'. These aren't for sale on your site but positive or negative transaction sums can be entered against them in the Admin Zone, so your profit/loss and cash-flow charts can be accurate.
Product kinds fit into the following general categories:
- Invoice: requests for payment created from the Admin Zone, paid for from the site:invoices module (above: work)
- Subscription: one-off purchases made from the site:purchase module (above: usergroup subscriptions)
- Shopping cart: an item purchased from the site:catalogues/site:shopping modules (above: catalogue items)
- Other (above: interest, tax, wage, other)
- Purchase wizard: one-off purchases made from the site:purchase module (above: none available out-of-the-box)
In the code these are defined via a PRODUCT_* constant from the ecommerce code file.
Whilst the abstraction of this may be confusing, it is very beneficial as all kinds of automatically handled products may be programmed to integrate with ocPortal functionality.
- a product that increases the number of available hits on banners
- a product that sends out an e-mail to a dispatch warehouse instructing them to mail out a copy of a bands latest album to the address specified in the member profile of the purchaser.
In more detailThe eCommerce system has a central transactional framework. Each type of product is sold by putting through transactions that have three key things that correspond to that product:
- product name (a codename, not intended for humans to read)
- product ID
Actual code (for programmers)This is an abbreviated line of code from within the get_products method defining a product in sources/hooks/systems/ecommerce/usergroup.php:
The 'product name' would be USERGROUP<id>. You can see how we have created a separate product for each different kind of usergroup subscription.
The 'product ID' is actually specified via the set_needed_fields function for most product hooks. In the case of usergroup subscriptions there is no set_needed_fields method because the ID of a row in the subscriptions table will be used in this case (subscriptions are a little different). i.e. it provides the information needed to link the purchased product back to a member's actual subscription record. Actually, the 'product ID' is always a string, so it has gone through the PHP strval function, and gets converted back via the PHP intval function.
You will see we essentially have two kinds of ID here. There's an ID embedded into the product name, and the 'product ID'. Typically we generate different product names when the prices will be different in each case, while the 'product ID' typically is either the purchaser's member ID or is the ID of another table that defines the specifics about the member's customised purchase. So in this case the different configured usergroup subscriptions in ocPortal have different prices, so are different products – but the ID of the actual purchased subscription is not price-determining. It is possible you won't have both – for example, the 'product name' may fully distinguish the purchase – in this case you probably should just use the purchaser's member ID as the 'product ID'.
Custom product hooks that are selling many individually priced items related to a specific user will often have a get_products function that lists a large number of items. There is a mechanism for get_products to only list products matching a specific 'product name' / product title, for performance reasons. If you have 1000s of products (or more) and are linking to purchase from custom code then you should return no products unless a filter is requested, and then use that filter to determine what products to show.
ocPortal basically flows like follows:
- User goes to the purchase module in ocPortal, and selects a product. Alternatively some custom code or link may bypass this step and direct them deep into the purchase module, i.e. to a known product.
- Product message is shown, if the hook defines a get_message method.
- Product agreement is shown that the user must agree to, if the hook defines a get_agreement method.
- User details what 'product ID' they are purchasing. This happens by them filling out a form determined by the product hook's get_needed_fields method. Actually for manual transactions in the Admin Zone there may be a get_identifier_manual_field_inputter method too, because admin's typically need more control when putting through a manual purchase. get_needed_fields does not have to be defined in a hook, so this step is optional too.
- A 'product ID' is generated via the hooks set_needed_fields method. If there was a get_needed_fields method then this typically works by taking the form input from the previous step and saving it, and getting an ID. However sometimes hooks may just generate an ID via some other known mechanism, or will just return the purchaser's member ID. It very much depends on the necessary purchasing process for the particular kind of product.
- An availability check is performed, if the hook defines a is_available function. This can do things such as stock checks, sanity checks, and permission checks. It returns one of a number of possible ECOMMERCE_PRODUCT_* values, ECOMMERCE_PRODUCT_AVAILABLE if purchase may happen.
- The user is directed to go pay off-site
- An IPN (i.e. background message) is sent back to ocPortal when the payment has happened, and/or when the user clicks to return back to ocPortal.
- ocPortal interprets the IPN to get back the 'product name' and 'purchase ID'. This is very hook-dependent. For PayPal it will look at the human item-name, find the matching 'product name' (i.e. codename), and it will already know the 'product ID'. For WorldPay it will have to look up a specific transaction in the trans_expecting table that ocPortal used to track the purchase, because WorldPay doesn't route through full information as PayPal does.
- ocPortal looks in all the product hooks to find a matching defined product, checks the price, then dispatches a call to the specified handler function for that product (e.g. handle_usergroup_subscription).
- The handler function then does what is necessary to make the purchase happen. For a usergroup subscription this means putting the purchasing member into the appropriate usergroup.
Subscriptions and invoices have some custom handling. Subscriptions can also end, so they are separately tracked with life-time details, and there are cancellation signals that are also fed through. Each separate invoice is not given a separate product even though each has a separate price, as it would get potentially too out-of-hand, so the handling of invoice price checking is hard-coded. Invoice products could actually be taken advantage of for this – you could generate custom invoices for particular purchasable items, if very large quantities of potentially differently-priced items are involved.
Manual transactionsNormally a transaction would go through via PayPal/WorldPay/SecPay. i.e. ocPortal sends a user off to PayPal, and then PayPal calls back saying the transaction has happened.
However, we've designed it to be modular. In ocPortal another method of doing a transaction is the 'manual transaction'. The manual transaction form just allows you to simulate a transaction; the idea is that you've received a cheque (or SMS payment, or whatever) and are manually telling ocPortal about it so that ocPortal can dispatch the sale. To perform a manual transaction you need to:
- access the manual transaction screen (Admin Zone > Audit menu > eCommerce link > Manual transaction icon).
- select the right product
- enter the correct ID number that the product will use to dispatch for this sale
- enter the sale amount that the product expects to be receiving (which you can leave blank for it to work this out itself).
Adding a capability for a usergroup subscription
Usergroup subscriptions are implemented on top of subscriptions. Set up a usergroup description as follows:
- Add a usergroup that members will be able to subscribe to. Make sure it is not 'Open membership', otherwise people could join it without paying.
- Assign additional access to this usergroup (or remove access from some of the other usergroups!).
- Go to the Setup section of the Admin Zone, then the eCommerce icon, then the 'Add usergroup subscription' icon
- Fill in the form. Be aware that a '5' 'monthly' subscription means someone pays 'every 5 months'. It does not mean a subscription is paid every month and lasts for 5 months – subscriptions last indefinitely until the user chooses to cancel them.
- Provide a path for members to visit the purchase page so that they may activate their subscription. By default this is possible via upgrade links shown in the logged-in box. You do also this via the site:purchase page-link, which can be inserted into a template, Comcode page, or most simply, onto your menu using the menu editor.
You can add as many usergroup subscriptions as you like. For example, [by adding more subscriptions] you could make it cheaper to subscribe yearly than monthly.
Invoicing a member
Members may view their invoicing history and staff may quickly locate unpaid invoices and invoices for which the work has not yet been fulfilled.
Online storeThe ocPortal online store was introduced in version 4.2, and works via the existing ocPortal catalogues system. Out-of-the-box there is the shopping cart catalogue (called 'products'). This catalogue has some hard-coded fields:
- Product title – the product title / name
- Item code – a code of your choice for the item, probably taken from your internal labelling system
- Price – the item's price (pre-tax)
- Stock level – the number in stock
- Stock level warning point – the point at which you should be e-mailed if the stock level reaches
- Whether stock is being maintained – whether you want stock control (as above)
- Tax rate – the tax rate, a percentage chosen from a list (you may edit the catalogue to change what is shown in the list)
- Weight – the items weight, in units of your choosing (but all items must consistently use the same units); the weight will operate with the config option that specifies how much it costs to ship a unit of weight
- Description – a description for the item
By default the catalogue is configured to allow product reviews, but you may turn these off if you wish.
Once you are happy with your set of fields, you can add categories and entries to your catalogue. Once you have entries, and your eCommerce configuration is configured, users can start using your store. Point them to the catalogue to do their shopping, and from there they will be able to navigate to the shopping cart and finalise and pay for their order.
Be aware that once ordered, a catalogue entry shouldn't usually be deleted. You can unvalidate the entries instead. This is vital to your ledger integrity. As of ocPortal 9.0.7 you can forcibly purge an item from the database and all past order history for it, but it's not usually recommended.
You may customise the appearance of the store in the usual way of editing catalogue templates (the 'products' catalogue already has most of the templates overridden with eCommerce-tuned styles, so you do not need to worry about accidentally altering the look of non-eCommerce catalogues).
From the catalogue you can add the item to the cart, and go to the cart (site:shopping module) to finalise and then pay for your order.
When orders are purchased, an e-mail is sent out to the staff so they can fulfil the order, and the stock level is adjusted.
Orders may be reviewed by the purchaser from their site:orders module. Order status can be changed (e.g. to dispatched) from the Admin Zone.
The ocPortal store has intentionally been kept lightweight. We're not trying to compete with the dedicated eCommerce systems on features, we're trying to provide something with fantastic ocPortal integration and extendibility that meets the majority of people's needs. This way we are able to compete by providing eCommerce as a part of the full ocPortal package, rather than a standalone eCommerce system that is bloated in it's own way but lacking the features needed for a wider website.
Deleting shopping cart itemsYou can delete any catalogue entry that is not present in an order. If an entry is in an order, you have to unvalidate it instead (unselect 'Validated'), and/or move it into an archive category and/or set the stock to zero. However, if the order was not purchased and the order times out, you may then delete the entry.
Advanced tax and shippingTax and shipping calculations have intentionally been kept very simple, but programmers may extend them. We intentionally have architected the calculations to go through some simple common functions that you can override to addon additional behaviour if you need to.
Shipping addressesSome custom profile fields are pre-installed to allow members to set their shipping address (instead of having to define it when they pay on the payment gateway). However by default these fields are not enabled, so you would need to edit them to enable 'Owner viewable' and 'Owner settable' for each field.
GuestsGuests may make purchases, if they have the right ocPortal permissions.
Guest users have no member ID, so ocPortal will store the order against their 'session ID' instead. Session IDs work using cookies, secured against the user's IP address. The session can be lost in many circumstances:
- if the user has their browser set to empty cookies when closed
- after the session expiry time (the default for this is 1 hour, but it's a config option)
- if the user's IP address changes (router's can sometimes do this, or ISPs may expire them routinely)
- if the user switches web browsers or machines
We can't tie the purchases to IP addresses because often thousands of users can be on the same IP address.
If you think users may be making delayed purchases as guests, you may want to consider removing Guest permissions to the shopping module, to force logins.
CurrenciesThe currency the website works in is defined in the configuration. If you use PayPal you need to make sure your account can accept this currency and is not set up to allow people to choose which currency to pay in (as it stops ocPortal being able to verify the payment if they use this feature).
A profit/loss account for a fresh install
ocProducts does not accept any responsibility for the financial accounts of users of the ocPortal eCommerce system. Accountancy features are provided as an aid only and are not guaranteed to perform to match any legislative, tax, or accountancy conventions.
ocPortal can generate the following charts over a custom specified period:
- Profit and loss account
- Cash flow diagram
Inputting a manual transaction
Manual transactions may be a bit tricky to perform at first, as it is necessary to enter the ID number associated with the transaction type in order to finalise it; typically the ID for a product is the member ID of the member who the purchase is attributable to.
Manual transactions may also be used to input transactions that are totally external to ocPortal, and thus otherwise immeasurable by the charting. For example, an organisations expenditure may be recorded by manually inputting a negative transaction for the special 'Other' product.
- The most popular online payment gateway
- Payment gateway
- The infrastructure for performing advanced financial transactions
- Instant Payment Notification
- A system, originally created by PayPal, for determining the progress of a financial transaction or ongoing subscription.
- An item in the eCommerce system that defines everything relating to the natural view of a product or service, including handling of any actions associated with a sale being completed or retracted; these products are not like supermarket products -- they are very engrandised and dynamic