Note: These instructions are specifically for cPanel hosted websites.
cPanel subdomains are a bit different then standard subdomains done via htaccess. Here are the steps to make a forum work as a subdomain with cPanel.
1) Go into cPanel, and “Create a Subdomain”. Set it to forum.yourdomain.com. cPanel will want to create a directory for you at home/public_html/forum. This is a good thing.
2) FTP into this newly created directory, and follow the directions for Running Multiple Domains or Subdomains or Subsites the “classic” way.
3) You should now have identical sites at www.yoursite.com and forum.yoursite.com. The trick now is to point all the subdomain requests to www.yoursite.com/index.php/forum. There are many techniques you could use for URL rewriting, but here’s a simple .htaccess file that will move all requests
RewriteEngine on
RewriteCond $1 !^(index\.php) [NC]
RewriteRule ^(.*)$ /index.php/forum/$1 [L]