How the compendium works... from a tech side?

[froodo]

Hi all! 

Some time ago i already said of my project of making a brazilian portuguese version of ChronoCompendium (nicknamed "ChronoMania), and my intentions of making a website similar to yours.

I was able to launch the forum part, but after having several struggles to emulate the another parts of the compendium, and i simple give up and just keep the forum.
My blatant issues was, firstly, how to make a layout persistent (or at least, similar) across all CMSs.

This part was partly resolved: i established all layouts for every cms need to be based in bootstrap v3 (to make easy of "normalize" all differences).

But isn't just that: I dont know how could be the best approach to "integrate" the forums and the homepage, specifically the News feed posts being selected forum posts being showed outside the forums (which seems like to be operated by SMF). Other thing is if the accounts are or not the same to access both parts of the site (forum and wiki, or perhaps the home too).

Some time ago i read in some place here the homepage engine was changed for something perl-related, which crawl mediawiki to show Term pages and the forums to show the latest news. I never seen anything similar before.

So my question is (of course only if its desirable by Zealty and other admins): Are there any documentation about the inner workings of the site? I looked across all About page and found nothing about this.

Additionally, can i contact any admin to ask some questions about these issues? To cast some light for the developement of my site.

I'm truly sorry if this thread are very unwelcome and against rules.
Peace for all and long live to Chrono series!

[Ramsus]

I wouldn't use SMF for a new site. We use it because the site is over a decade old. I might consider Discourse these days. You can integrate accounts through Single-Sign-On (SSO):

https://meta.discourse.org/t/official-single-sign-on-for-discourse/13045

The homepage is currently just a PHP script that uses SMF's db functions to query for news posts and render them. We used to have a Python script that queried the database directly and cached the results with memcached. We use forum content to drive the news page, because it's easy for the admins to handle and provides a clear places to comment on news posts.

Wiki content is served through a PHP script that actually pulls content from MediaWiki through it's JSON API on the server side, since I got sick of dealing with MediaWiki skins. Content is edited at an alternate URL where the wiki is served directly with its default skin.

I would not use MediaWiki for new projects. We still use it on this site for legacy reasons, but we found that an editorial workflow worked better for us than an open wiki workflow, so MediaWiki was honestly never a good fit.

Accounts were linked for a while between MediaWiki and the forum accounts using a plugin. However, unified accounts weren't really important in our case since the ratio of forum members to wiki contributors ended up being very low.

Also, open wikis work best with large numbers of dedicated maintainers and contributors, otherwise the ratio of spammers and bots to people can easily become unwieldy. Think about this problem early, and how you plan to deal with it. Also, set up automated backups and practice fully restoring your site and its content, because shit happens.

Finally, if you really want the integrated look and feel, you really should just build your own system using a framework and reusable components. Ruby on Rails is a great way to do so without much work, but it has a steep learning curve to use it well (i.e. to understand all its magic) and can be tedious to deploy at first. It's what I would use if I wanted to do something with minimal time investment since I've used it professionally before. Django (a Python web framework) is more consistent and clear, without any magic. However, the Laravel framework is probably the easiest for beginners with some PHP and traditional website hosting experience to approach and play with.

Read The Twelve Factor App and OWASP Secure Coding Practices, and keep all your source code in version control. Standardize your development environment with Vagrant or create a virtual machine image that you keep backups, and script your deployment process so that pushing out new versions of the site's backend requires a single command. Also, start thinking about how you can allow others to contribute to the site's backend.

[Ramsus]

Also, we use nginx and php-fpm running on a Debian server, with haproxy in front running as a reverse proxy and SSL terminator. Hosting is currently on a bare metal dedicated server, but we'll be moving to VPS hosting on Digital Ocean to ease maintenance.

If you're not familiar with Linux system administration and setting up/securing servers, you should look into shared hosting and building something with Laravel.

[Ramsus]

Some other advice:

• Use Markdown for your forums and wiki content:
  
  https://guides.github.com/features/mastering-markdown/
  
  This makes your content rendering consistent and more future-proof.
  
  CommonMark is a Markdown specification with lots of ready to use parsing and rendering libraries.
  
• If you use a database like MySQL, write a wrapper around your DB interface that isolates the specifics, so you can migrate to other databases by just rewriting one or two modules or classes.
• Use a dependency manager. Ruby has gem. PHP has Composer. Understand Semantic Versioning.
• Don't use Node.js unless you're a JavaScript guru who knows how to use closures and Promises, and don't buy into trendy bullshit. Asynchronous, event-driven programming gets difficult with larger projects, and you can easily end up with a huge mess of poorly organized and hard to follow code. That said, Node.js with Express is hard to beat for building web APIs.
• Learn where your error logs are and how to make sense of them. Make sure they're set up correctly, too, and don't expose detailed, developer stack traces and error pages to the public!
• Check out Caddy. Otherwise, stick to nginx and php-fpm running on Linux.
• Don't forget your firewall. Learn how to set up a stateful firewall.

[Ramsus]

Oh, and set up a virtual machine running the same software as your hosting environment, save a backup of it, and then deploy and test everything you do on that before putting it into production. This is called "staging" the site, and helps a lot with catching dumb mistakes.

[froodo]

Well, seems my limited knowledge, money and time are pointing me to the wrong (and messy) direction...

First, and main problem: Developement isnt my field. I work mostly with support and hardware, and in fact i divert from dev some years ago, after having personal struggles in high (and also technical) school. Today develp and related is just a hobby, and most of my skills are in web design and a tiny PHP knowledge. Outside of web, i have some experience with PAWN (the language SA:MP - GTA SA Multiplayer) uses.

Back to the topic: i'm using a cheap rented hosting, aprox. US$ 3.76/mo, which only have PHP and MySQL support. No Ruby on Rails in sight. Other hosting (with more options or space/bandwith) was considered in the past, but give my conditions, its outside of my reach.

Second, the CMS to be used: My initial idea was to serve the page using three cms: one for the frontpage (Wordpress), one for wiki (MediaWiki) and another for the forum (PhpBB). I tried to minimize the need for multiple CMS searching for plugins for both Wordpress and Phpbb which allow wikis and forum/frontpages, but while wordpress only have a freemium Wiki plugin and the very limited bbPress (which are semi abandoned and without any documentation), PHPBB v3.1 dont have any type of portal or wiki plugins for its version, and i doubt any developer would port the previous version MODs to current version extension system.

In fact, one of the reasons that made me re-consider the 3-module design was that there was a MediaWiki skin which uses bootstrap as base. Thus i can easily adapt some parts of the layout to give a harmonic and similar appearance across all the three CMSs. The other was the frustration of having a member who has the knowledge to feed an encyclopedia but does not have a proper place for it.

Changing the planned CMS would be a good thing, specially to use Markdown as forum syntax. But the folks of PHPBB dont think Markdown is a good thing.
Another but annoying issue is the freemium nature of my current Bootstrap-based PHPBB style. Despite being MIT-licensed, the author refuses to add some basic compatibility with the more used plugins, and the layout isnt that good. I struggle to make significant changes (aside of the header of page) and several bugs are still present.
But, the major headache to switch for other CMS is the (in some of them) difficulty to tweak the graphical style of it, and of course the type of database/language used. If the system have easy theming and/or some free theme/style based on bootstrap (at least for reference) and whose requeriments meets what my current hosting plan offers, i can give a try.

Third, the mobile access. Most people blame my site failure to reach a pretty decent base due to this. In fact one of the issues is a tendency of people to get lazy and stick only in social networks, crippling the audience of the blog. Another are the limited bandwith  for mobiles here on Brazil (since i launched the forum my cousin arent able to access it), and the demand of a standalone app for Android/iOS/WinPhone, something i dont have any clue of how develop.

Another insignificant (for this topic) problem is the haters i've collected across social networks during CM's passage there. People only consider Trigger and hate Cross, most of them because cant read english or dont know how setup propely a PS1 emulator.

Lastly, i dont have time and knowledge (as stated before) to learn a new language or platform and in a conceivable short time began building a so large and complex system. I never developed something so great, and alone i felt its almost impossible.

And hiring a freelancer are too expensive to me. But it would be a great solution.

Perharps my plan, for the present moment and future, is very ambitious but pointless, given the small (and decreasing) size of the fandom even on Brazil.
Maybe my efforts are clueless, but as i stated before, it was my dream in first place, because i always wanted to see something like this place full in portuguese to have my friends (and other fans of the franchise) keeping around the flame.

I have some friends wanting me to play other RPGs or non-RPG games and make something similar to ChronoMania, but it isn't what I want.

Well, the lesson I learn from it all is that I need to choose priorities for my community, and even for me. Getting the forum working was a hard work, but i want to improve and perfect it, and this is why i poke annoyingly brilliant people (like you all) to give me clues about anything.

Anyway, thank you so much Ramsus for the advices. And of course for keeping this site alive and rocking with Zeality for all these years. You're superb!

[Ramsus]

You'd be surprised how far you can push a $5/month Digital Ocean VPS:

https://www.digitalocean.com/pricing/

But yeah, having a friend who can help handle the technical aspects of building and hosting the site would be your best bet. Zeality and I started this site when we were still in high school.

Don't waste your money on a freelancer. Fan sites should be worked on by fans and as a labor of love. If you have money, save it to pay for hosting costs.

Whatever you do, always have a migration plan. I ran the Compendium off an old desktop computer running FreeBSD using a broadband cable connection for an entire year and a half when I was in college, because we couldn't afford a decent server.

Anyway, if you just need a place to start, start with a wiki and a forum, and don't worry about integrating them or making them look the same. phpBB is a good choice for forums -- I'd prefer that over SMF if I had to use something written in PHP.

I never liked MediaWiki. It's a real resource hog and not really geared towards customization. Check out DokuWiki instead.

Don't bother with Wordpress for anything. Just use the wiki for your homepage.

Try to avoid plugins and extensions -- very few are ever maintained properly, and you'll always be updating them.

[froodo]

First of all I thank you from the bottom of my heart, Ramsus, for the advices and tips, especially the last answer.
I began to test DokuWiki, and this little CMS is simply sensational. Very flexible and customizable. I quickly found a bootstrap theme and some essentially useful plugins, incidentally, Doku was in general far more easier to configure than phpbb (and obviously MediaWiki).

Well, now I am seriously thinking of structuring the site as follows:

Code: [Select]

.
├── the index script (similar to how Compendium works, it extract a x number of posts from a certain phpbb thread and show it)
├── forum
|   └── continue to use the same phpbb extension already deployied
└── wiki
    └── dokuwiki

Additionally i will create a folder called "assets" (and assign to a subdirectory, like assets.website.com) to store the common layout files for all themes to prevent duplicates.