[fChk]

From what I've read recently memcached may be better for me at the present. The system requirements to run redis seem a bit steep for a small fry like myself. From the redis docs, 4GB ram is recommended for development; for a production environment start at 15GB and 3 nodes. From your experience, would you say these recommendations are appropriate? Such a set up is unfortunately beyond my means at present.

Memcached, from what I've read seems a lot more forgiving and flexible in terms of system requirements; I believe I could set up a development environment with the tools I have on hand, which is a big perk for me.

Obviously it'll all depend on your traffic load and the popularity of your WebApp. Indeed, Memcached is more lightweight compared to Redis, given that Redis has more features than Memcached, including data persistence.

There are pros and cons for both technologies and it all depends on the use case for chosing the best one. Memcached store data in memory and is volatile (no filesystem caching); it's best feature is that it supports clustering while Redis doesn't (it's single machine as of version 4.)

Redis best features is that it has data persistence while functioning in operational memory with blazingly fast performance. It can even be used instead of MongoDB as long as the data model stored is fairly simple.

For development purposes, I would advise you to try to get one of Post4VPS VPS-9 and do your live testing in there; it should have all the resources you would need. If not, a VPS with min. 4GB RAM, 2 vCPUs might get you going...

As a side note, Apache Cassandra is yet another in-memory key-value stores that has clustering capabilities

I did a bit of browsing at w3schools and the nodejs web server looks fairly straight forward. I think it would be a relatively simple matter to port my php scripts over to nodejs within a few days or maybe a week.

Not so sure about that :-)

You need time to get used to Node.js and its ecosystem including the NPM package manager tool and its library of packages in circulation.

I would imagine that you won't code anything from scratch (as I did 6/8 years ago) and that you'll be using the existing popular NPM packages to get the job done (eg http-server, express, websocket etc...)

You see, the first thing to do is to learn the fundamentals and core concepts of Node.js, once done you need to get used to 'Express' which is the most popular web application framework for Node; or 'Backbone.js' which creates an MVC model around the http-server etc...

In my mind the process has a learning curve, at least, to get used to all the APIs that those packages will provide you to build your own scripts on.

This is why I only use Node.js everything else around it is my own code that I maintain with every iteration of Node.

This is also why, I think, it's better to finish that project in PHP before switching to Node.js (which is an entirely different state of mind.)

I'm curious what techniques you used for your node web server; if there are any particular pitfalls or caveats to avoid; or anything you'd specifically recommend. Perhaps that would be a topic for another thread?

Not sure if I understand what you meant by 'techniques', but if you meant the use of the old-style based on call-backs vs the more recent one based on Promises, then my response is both. Initially (before the advent of Promises in JavaScript) it was all call-back-based (the famous Callback Hell!), but did some modification when Promises were introduced around 2015.

I'm planning to get a P4V VPS again and maybe I'll be posting more on Web development topics then. Up-till now the majority of my contributions here were done with my SysAdmin hat ON given the audience :-)

One thing I'm having some trouble with is finding what I would call a "good" tutorial for either memcached or redis. There's plenty of docs, which is fine, and I'll read them (grudgingly and slowly), but I'd like to find something a little friendlier to the uninformed.

I'll check my archives and I'll post few links for reference next week-end - I promise :-) .

I'll also PM you few ebooks for download from my end, if that's ok with you. I'm more of a book type of guy than of an Online reader -Old habits die hard!..

Books have structure and generally are easy to follow if one has the right motivation and matches the book's assumptions for its readership.

Online tutorials quality, on the other hand, varies widely and you need to really trust the source to be an authority on the subject of some kind... The majority are just copy/paste of copy/paste of... of some authority(/or reference doc.)

[fitkoh]

I think I'll leave out the quote so it doesn't turn into a wall of text

In terms of traffic, it's hard to predict - for right now the only users are me and my flatmate, and it runs like a dream just using php files served over http to interact with the database; but I can definitely imagine performance to decrease with a lot of concurrent users. However, as I've said, I designed the client with minimalism in mind. All player interactions are turn based, and database queries are timed; but in a dream scenario of 1000s of concurrent users, it probably wouldn't be enough.

I think there is some wisdom in what you advise: finish it first in php, and then add on what's needed, and change what can be improved. So for now, I think I'll stick with the traditional relational database and php for the backend, and focus on getting just these 2 pieces working as well as possible. If performance demands it, I can work on swapping php for nodejs. If performance demands more I turn to memcache. And if I'm fortunate enough to build a large community of paying users, I can look into redis.

I've already converted a few of the scripts to js - but I can see what you mean about a different mind set. In php, I was very careful with a few things: to sanitize all the inputs, as php is known for injection vulnerabilities.  I made sure to place lots of exits in every script: if at any point anything goes wrong, the connection drops and error returned. None of this was too complicated; it just involved some learning.

But js is proving more difficult. The language and syntax feels much more complex than php. There's uppercase and lowercase and punctuation everywhere. For me, it's quite simply hard to look at and read... but as you said, there is a learning curve. I'm  sure with practice I can improve. Nodejs recommends leaving the connection open and reusing it, as opposed to closing the connection - which in my mindset just seems odd, but I can see how it would get a little performance gain. The methods of sql injection prevention are very different and less well documented (unless I'm just looking in the wrong places - which is very possible). Although I do like that nodejs can run asynchronously, whereas php cannot.

Thus far, I've coded all of the server scripts by hand, and I'll probably do so for the immediate future, until I get my footing. I think it's important, starting out in the learning process, to code from scratch so I can get used to the syntax and structure of the language. At least until I can look at it and trace the flow of information in a script. Perhaps I'm wrong? Also, while it's handy to have ready made tools that do  what I want, it isn't always the most efficient. In my experience, it can lead to bloat and unnecessary complexity, so I'll try and stay away from libraries and packages until I have a better understanding of what they do and how to modify them to my needs.

Regarding your self-coded web server, it's a topic I never considered for myself (until now), and I'm curious why you thought it was necessary/beneficial: what purpose did you have in mind when writing it, and what methods did you use to achieve this purpose? Even your simple explanation has given me a few new terms to look up: while it might seem easy to you with your experience, it could prove an opportunity for learning for me (and others who might view this in the future)

We do have a few nice VPS here with specs that would probably handle live testing; but I'm hesitant to ask for it for a couple reasons:

1. As an app in development, my usage is very low, and I'd feel bad about having one of the best VPS we have and not getting high use out of it.
2. I like posting, and having the freedom to post. If I won a VPS here, and find it useful, then I'm locked in. My choice to post or not is now limited, and with consequences.
3. I feel like I'm still not quite ready for it: both in my knowledge/experience, and in my need. While I hate to admit it, my precious app could completely flop, regardless of the architecture it's running on. Is it really desireable to add on more and more technology/expense/work without any success to warrant it? I think not; but I'm an expert at making mistakes.
4. It's a risk. There is no guarantee that I'll keep the vps, even if I can win it. If someone comes along and makes more quality posts than me one month and asks for it, they might get it - and that would be fair. While currently we have an abundance of vps at the ready, that could change at any time. An influx of high energy posters could mean that our vps wins become highly competitive - much like the old days of freevps. Which would be great for the community,  but it'd suck for me to spend days/weeks/months setting up a redis system that's integral to my apps operation only to have it snagged from underneath me by a superior poster.

I'm not opposed to reading books; I'm fortunate to have an old kindle lying around that should be perfect for reading e-books. I tend to agree with you regarding the quality of web tutorials, which is why I specifically mentioned "good" tutorials. 

The big issue I have with tech docs (although I'll read em if necessary) is they often contain a ton of jargon and explain everything in the most verbose and complicated way possible. It can take forever to get through tech docs when you have to stop every minute and look up a new term, or read through a wall of text to get a concept that should be explained in a few sentences.

I really like the format w3schools is set up with; but it doesn't cover all the topics I'm interested in. It gives simple explanations for the smallest possible code fragments with examples, and demonstrates how those fragments can be assembled to work together with the minimal amount of jargon possible.

That being said, I enjoy learning and reading, so I'll likely examine anything you recommend. I can't guarantee I'll read everything but I will definitely give it a look - I'm going now to see what I can learn about Apache Cassandra and try to find out what an MVC model is

[fChk]

I think there is some wisdom in what you advise: finish it first in php, and then add on what's needed, and change what can be improved. So for now, I think I'll stick with the traditional relational database and php for the backend, and focus on getting just these 2 pieces working as well as possible. If performance demands it, I can work on swapping php for nodejs. If performance demands more I turn to memcache. And if I'm fortunate enough to build a large community of paying users, I can look into redis.

That's the way to go!.. The point for me to bring Nodejs+Redis into the discussion was to emphasize that the LAMP stack has its limitation and that you shouldn't over-complicate(/engineer) your App setup. Instead when you hit a bottleneck just fall back to the Nodejs alternatives.

I've already converted a few of the scripts to js - but I can see what you mean about a different mind set. In php, I was very careful with a few things: to sanitize all the inputs, as php is known for injection vulnerabilities.  I made sure to place lots of exits in every script: if at any point anything goes wrong, the connection drops and error returned. None of this was too complicated; it just involved some learning.

That's because PHP is at its core a procedural language, while JavaScript is an event-driven one. Big difference in how you should think about things.. especially for handling errors/exceptions asynchronously.. And that's exactly what I meant by the mindset that is completely different. With the introduction of Promises, the mental exercise became less painful of what it was when it was all callback-based.

But js is proving more difficult. The language and syntax feels much more complex than php. There's uppercase and lowercase and punctuation everywhere. For me, it's quite simply hard to look at and read... but as you said, there is a learning curve. I'm  sure with practice I can improve. Nodejs recommends leaving the connection open and reusing it, as opposed to closing the connection - which in my mindset just seems odd, but I can see how it would get a little performance gain. The methods of sql injection prevention are very different and less well documented (unless I'm just looking in the wrong places - which is very possible). Although I do like that nodejs can run asynchronously, whereas php cannot.

It takes time to get to it, and yes JavaScript (aka EcmaScript: ES) has evolved beyond recognition during this decade--up to ES2019; so you should check that out too.

As for connection handling in Nodejs, you just trigger the end method of the response Object with the corresponding argumets:
See: https://nodejs.org/dist/latest-v14.x/doc...g_callback

Thus far, I've coded all of the server scripts by hand, and I'll probably do so for the immediate future, until I get my footing. I think it's important, starting out in the learning process, to code from scratch so I can get used to the syntax and structure of the language. At least until I can look at it and trace the flow of information in a script. Perhaps I'm wrong? Also, while it's handy to have ready made tools that do  what I want, it isn't always the most efficient. In my experience, it can lead to bloat and unnecessary complexity, so I'll try and stay away from libraries and packages until I have a better understanding of what they do and how to modify them to my needs.

No!.. you're not wrong!.. And I'm glad to read this response because that's exactly my general advice to anyone interested in the Web!.. It's way too easy to patch third-party scripts together and hack your way through them But you'll never learn how things really work and the result will always be highly insecure and way too dependent on other people's work/will, thus unstable (most likely dead at the 2/3 version iteration.)

Regarding your self-coded web server, it's a topic I never considered for myself (until now), and I'm curious why you thought it was necessary/beneficial: what purpose did you have in mind when writing it, and what methods did you use to achieve this purpose? Even your simple explanation has given me a few new terms to look up: while it might seem easy to you with your experience, it could prove an opportunity for learning for me (and others who might view this in the future)

The short answer is because I could do it :-) At that time, I've already had a fairly full understanding of how the Web works: the server part (mainly Apache back then), the client part (W3C-compliant web browsers) and the language they use to communicate with each others -ie the HTTP protocol-.

With the availability of Nodejs, the fact that I've came to master JavaScript around those times and my rising need to use Websockets for real-time WebApps gave me the motivation to start  building an HTTP-server that speaks HTTPv1.1.

It took me 9 months to came up with an implementation that's Okay-ish and that has evolved slowly since then. But I must say that it's still missing few features that I don't need/use but gets the job done as intended.

An urgent update that I must attend to ASAP and that I've been postponing for quite sometime now is the support of HTTPv2, which is built-in as of Nodejs v14.x.x

4. It's a risk. There is no guarantee that I'll keep the vps, even if I can win it. If someone comes along and makes more quality posts than me one month and asks for it, they might get it - and that would be fair. While currently we have an abundance of vps at the ready, that could change at any time. An influx of high energy posters could mean that our vps wins become highly competitive - much like the old days of freevps. Which would be great for the community,  but it'd suck for me to spend days/weeks/months setting up a redis system that's integral to my apps operation only to have it snagged from underneath me by a superior poster.

The main way a VPS is lost is when one misses the monthly post quota; the other ones are related to some form of breach (ToS etc..) And, yes, that's always a risk!

As for posts quality, no worries you're for-sure up to the challenge :-)


Few links related to Redis+Memcached:
A lot of the bookmarks generated 404, so here is what's left:

http://memcached.org/
https://www.digitalocean.com/community/t...centos-vps
https://www.pontikis.net/blog/install-me...-archlinux
https://www.cyberciti.biz/faq/rhel-fedor...ystem-rpm/

https://redis.io/documentation
https://github.com/TheDeveloper/redis-session-php
https://rtcamp.com/tutorials/php/redis-php-sessions/
https://www.digitalocean.com/community/t...untu-14-04
http://highscalability.com/blog/2011/7/6...redis.html
https://www.digitalocean.com/community/t...untu-14-04
https://www.sitepoint.com/saving-php-sessions-in-redis/

Good luck!