Skip to content

Instantly share code, notes, and snippets.

@anushshukla
Last active September 5, 2023 00:35
Show Gist options
  • Save anushshukla/ac686097ea06b1def826b44dda1209f1 to your computer and use it in GitHub Desktop.
Save anushshukla/ac686097ea06b1def826b44dda1209f1 to your computer and use it in GitHub Desktop.
Software Building Guidelines

Index

Software Application Basic Features

Applications should have

  • Gamification: Better User engagement
  • Shopify: Business to grow
  • Internationlisation: Globalisation of the application
  • Localisation: Better UX
  • Chat: Easy communication

Cache Policy

Working

Key-value pair like a hash map with O(1) complexity.

Requirement

Performance optimisation to improve UX by bettering the speed of the server response time to the client.

Pre-requisites

Cache data which is being fetching at regular intervals and frequently which changes rarely

Tool

Redis

Examples

  • HTTP GET Requests
  • Database fetching queries
  • Tokens
  • Rate Limiter
  • Chat module

Programming Standards

Naming convention

With reference to Naming convention standard

snake_case

  • column_name (SQL)

camelCase

  • Variables
  • Function
  • Methods
  • Properties
  • Packages
  • nameSpaces
  • collectionName (NoSQL)
  • fieldName (NoSQL)

PascalCase

  • ClassName
  • Interface
  • TableName (SQL)

UPPER_CASE

  • Constants

kebab-case

  • CSS Identifiers

Project Dependencies

Choose them wisely

Keeping the publishing date in mind, we need to consider the following factors

  • Maintainers: More the maintainers => Deprecation prevention
  • Releases: Shorter the frequency and at regular intervals shows => Stable
  • Size: Smaller => Less heap memory consumption
  • Issues: Less open issues with low severities => Safe and Secured
  • Starts: More the better => Shows the popularity
  • Major version released => Matured
  • Building your own package is feasible and benefically considering the reinvention
  • Check Roadmap
  • Check support => regular issues fixes
  • project dependencies maintenance (periodically updated, especially semver) to
    • avoid breaking changes
    • avoid deprecation
    • receive support from maintainers
  • project dependencies auditing to avoid and fix security vulnerabilities

Development Guidelines

  • Design Patterns
  • Correct Requirements (Specs & UAT)
  • Modularity
  • Multi Programming Paradigm supported programming languages
  • FP
  • OOP
  • Implement SOLID principles for FP / OOP
  • Implement SQL databases if ACID properties are required
  • Implement NoSQL databases if BASE properties are required
  • Source code should
  • be less (unnecessary functions and variables must be avoided)
  • by modular (re-usablity)
  • be understandable (short, simple and meaninful naming conventions)
  • be formatted
  • have informative logs
  • have error logs, especially where the error is suppressed and silent
  • be documented using developer comments
  • Development Environment
  • Environment
  • Strict (application should crash even in cases of warning)
  • Verbose logging (nothing must be silent / suppressed instead should be logged to have it fixed)
  • Development Practice: TAD -> TDD
  • Global .gitignore
  • .env file to maintain environment specific values
  • server hot reloading on file change feature must be present
  • interepreted environment (only during development-like environments) for saving time
  • compiled build (only for production-like environments)
  • .env implementation -> Simpler environment setup
  • linter implementation -> Better coding standards
  • linter with strict rules (specially for dynamic typed languages and especially for loosely typed languages)
    • avoiding code complexity
    • improve code quality
    • preventing obselete LOC / unwanted pieces of code
  • implement formatter for better code readability and comprehension
  • write tests for better code quality
  • coding
    • Add // @todo comment wherever needed, especially for unused or commented code blocks
    • Repeative code blocks must be moved into a util function or a class method
    • Repeative if-else code blocks can be looped (subjective to the code block)
    • Neagtive cases must be checked at the beginning of the code to stop further execution
    • Require / Importing of files should be absoulte paths relative to root directory of the codebase (depends on the programming language)
      • variable names should be prefixed is, can, should, etc when evaluating and storing a boolean expression to it
    • Failsafe try-catch or some error suppressing library should be written for unknown property access of an object (like JSON)
    • Prevent Memory leaks by avoiding pass by reference and using pass by value while using immutables instead of mutables (more like FP) but do use it otherwise in OOP when you are sure of it
  • engineering
    • CI -> Risk Mitigated Deployments (Tests + Code quality check)
    • CD -> Easy deployments (Dockerize + Kubernetes)
    • Ansible -> Dynamic Development Environment
    • Project dependencies Audit Report-> Security vulnerabilities fixes
    • Project dependencies Updation Periodically -> Prevent deprecation or breaking change upgrade
  • SVN
    • commit hooks must be present to prevent invalid commits (poor message, lint / prettier check, etc).

Optimisation

Application (Backend)

  • Unwanted loops
    • break, return and throw in case of negative cases
    • continue to skip the ones not required
  • Logs
    • Use debug logs for development which can be disabled in higher environments
    • Avoid logging complex data structure
    • Avoid logging big data structure
  • Unwanted queries
  • Unwanted business logic code
  • Remove unused variables and save the risk GC pausing the main thread.
  • Execute asynchronours operations in parallel which are not inter-dependent
  • Execute independent asynchronous operations in the background (worker thread, message queue, cron, etc) with logging
  • Database (SQL / NoSQL)
    • Queries
      • Filtering: columns / fields are indexed (composite / compound index also works if applicable)
      • Selection / Projection: Unwanted columns / fields to be removed
    • SQL
      • Missing join query as in 2 related tables queries are executed separately.
      • Table join column is indexed and hopefully an integer

Infra

  • Caching like AWS CloudFront for frontend application builds and public assets (loogs, images, etc.)
  • Total Nodes in Redis Cluster = n (shards) * (replication factor) + 1 (master)
  • Insert / Update -> SQL
  • Background -> Related data save from RDS to NoSQL
  • Fetch -> NoSQL fallback to SQL
  • AmazonElastiCache -> AutoScaling Redis
  • Keeping Read and Write database separate.
  • Read database -> Clusters
  • CDN AWS CloudFront for
    • Public Assets
    • Frontend Application Builds
  • Logs archive to AWS S3
  • Back SQL Database like AWS Aurora backup to Cloud Storage

Database Guidelines

SaaS Guidelines

SLA (Service Level Agreement)

  • Issues resolution TAT
  • SLO (Service Level Objectives)
  • Uptime / Downtime
  • Recovery Point Objective (RPO)
  • The Recovery Time Objective (RTO)
  • Version Retention Objective (VRO)
  • Geographic Redundancy Objective (GRO)
  • Latency
  • Packet loss
  • SLI (Service Level Indicators)
  • MTBF
  • MTTR
  • MTTA
  • MTTF
  • Code minification to minimize the build size.
  • Implement SPA (single page application) for quick performance.
  • Implement SSR (server side rendering) for deliverying better SEO (Serarch Engine Optimisation).
  • Atomic design methodoloy implement to keep code modular & reusable.
  • Smart/Dumb components implementation for UI & logic.
  • Static media files should be effectively compressed & hosted at Cloud Storage behind CDN caching it for fastest delivery.
  • Eligible network requests caching on client machine with appropriate policies in place
  • Maintain & communication encoded data when suitable to reduce payload size over network for quick throughput & keep client machine storage light
  • Stream large data to avoid running out of memory & improve UX.
  • Buffer small data for quick loading
  • Fetch appropriate static media files as per client machine's screen size & resolution.
  • Parallel processing of independent asynchronous tasks.
  • Background processing of non-critical tasks.
  • Create themes for customisation.
  • Lazy load non-critical resources like media files to avoid page load performance issues.
  • Eager load page components to avoid page load performance issues.
  • Implement pagination/infinite scrolling to avoid running out of memory & improve UX.

Infra Improvements and Performance Optimisations

FE

FE UI/UX & Performance improvements in regards to page load and interactions

  • Implement memoized functions for quick computation
  • Paginate data using virtual / infinte scroll for better UX (speed and memery performance)
  • Eager Loading of static files like JS, CSS and HTML for the required page
  • Lazy Loading of non-critical assets like images, videos, etc.
  • Memoize functions for quick computation
  • Decrease Response Size by implementing GraphQL
  • Use PNG for static images requiring transparent background but with proper resolution to reduce file size for quicker loading
  • Network Caching with proper cache-control request header and invalidation for API, images, videos, etc.

Web apps

  • Use SVG for icons for web graphics as they're anti-aliasing, easy to implementat & leightweight.
  • Use WebP with fallback to JPG for static images without transparent background for quick loading as they're leightweight
  • Virtual list implementation for large data sets to decrease DOM load
  • Use polyfill to support missing API in old browsers
  • Use shim to support missing API features in old browsers
  • SPA (Single Page Application) for better UX (speed performance)
  • SEO (Search Engine Optimisation) using Gatsby, Nextjs, etc for better page ranking by using SSR
  • PWA for offline app as well as mobile & desktop app support

DevOps

Data Layer

  • Indexing fields which are used to filter
  • Denormalised Data (Stored Procedures / Views)
  • Partitioning
  • Sharding
  • Replication (Clusters)

BE Optimisations

Recommendations

All the MySQL tables have

  • Engine -> InnoDB
  • Charset -> utf-8
  • Collate -> utf8_unicode_ci
  • Name -> PascalCasing

All the columns have

  • SnakeCasing
  • tinyint(1) for true / false
  • timestamp value is being used for date / time related columns
  • char instead of varchar if we're sure about the length of the string

Indexing

  • Only those columns which are being used in where clause quite frequently
  • Composite index keys are used instead of individual index keys
  • Foreign key constraints having no action on update and delete

Prerequisites

Prefix EXPLAIN to your queries to get an idea where indexes might be required!

Make sure that the columns being indexes are used quite frequently used at least in any of the following

  • JOIN
  • WHERE
  • HAVING
  • ORDER BY

Also, please make sure that the indexes are

  • NOT NULL
  • COMPOSITE (if eligible)

Check with EXPLAIN query to confirm if the indexes are working well

  • with SELECT and at the same time
  • not hampering INSERT / UPDATE performance!

Master tables have

  • deleted_at -> timestamp having on soft delete
  • update_at -> timestamp having on update & default as current_timestamp
  • created_at -> timestamp having default as current_timestamp

Mapping tables

  • no primary key required
  • no soft delete required

Migration queries

  • Make sure end of statment ; is not missed
  • Make sure database is either used consistently for all the queries or not if database is already pre-selected

Project Development Checklist

  • Refer PRD and reach out to Product Owners i.e. Product Managers owning that feature
  • Refer Designs if UI/UX changes are involved
  • Create and refer ERD
  • Refer legacy system if migration activity
  • Reach out to Peers for help
  • Reach out to Leads for guidance

FAQ

Q: What are the basics of programming language? A: Understanding of

  • declaration and definition of variable & function
  • data types and data structures
  • control structures
  • arithmetics operation

Q: Which programming language to choose? A: As per use case, For example

  • Real time Applications like Chats: NodeJS
  • DS developing AI using ML & DL like ChatBots: Python
  • CMS like WordPress: PHP
  • High I/O Performing Applications like Banking Servers: Go

Q: Which is the best hosting server? A: AWS

Q: Which is the best cloud storage? A: AWS S3 as compare to Azure and Google Prices SDK (Major Programming languages support) Archiving (using AWS S3 Glacier) Cache (using AWS CloudFront) Replication Meta (Object-level tagging)

Q: Which versions to choose when installing anything (generic question)? A: LTS or Current (if stable enough and LTS is just around the corner)

Q: Should we periodically upgrade to next major stable versions (project dependencies or programming languages)? A: Yes, otherwise security patches becomes our headache bug fixes leads to making code hacks desired new features are inaccessible major version upgrades leads to breaking changes with possibility of no reverse / backward compatibility

Q: Which is the best messaging queue software? A: Apache Kafka

Q: Which is the best server setup? A: While serverless are more granular and modular than servers. In the end, it depends on the use case, like these are the scenarios best for serverless.

Q: Which is the best server architecture? A: Micro-services is the best but Monolith and SOA have their own set of use cases for specific scenarios.

Q: Which is the best API Gateway? A: Kong

Q: Which is the best Website Application development? A: Vue

Q: How to ensure application quality? A: Put in quality checks like (considering javascript as the technology)

  • Code lint -> Eslint
  • Code format -> Prettier
  • Code quality -> SonarQube
  • Code tests -> Jest
  • Code review -> Leads
  • Quality build -> Devs (UT) / QA
  • Code performance -> QA (Automated Load Testing)

Q: Which is the best NodeJS Framework? A: Although it is as per use case but Express takes the cake. Over it, we can always built our own custom framework which is

  • Not strongly opinionated
  • Loosely coupled
  • Hooks
  • Listeners
  • Overides
  • etc.

Q: Which is best NPM ORM/ODM package for Nodejs Projects? A: TypeORM

Q: Which is best way to version data sources schema for Nodejs? A: TypeORM Migrations

Q: Which is the best NodeJS logger? A: Winston

Q: Which is the best NPM package for making http requests? A: Axios due to its Features

Q: What / Which is the best way to monitor the server? A: DataDog / Elastic APM with Kibana using supported APM Agent or New Relic.

Q: Which is the best error monitoring tool? A: Rollbar or Sentry

Q: What is the best way to test across different devices and browsers? A: BrowserStack and pCloudy and SauceLabs.

Pull Request Guidelines

  • 1 Assignee who is an Admin (Maintainer / Owner)
  • Minimum 1 Reviewer who is at least the co-author of the code being modified
  • Squash commits checkbox should be enabled by default (can be unticketed if the contributor has meaningful commit messages)
  • Source branches delete after merge checkbhox should be enabled by default (contributor should untick if the branch is required even after merge)
  • Minimum Approvals
    • Product Feature: 3 Approvals (1 Reviewer + 1 QA + 1 Product)
    • Tech Debt: 2 Approvals (1 Reviewer + 1 QA)
    • Engineering Tasks: 1 Approval (1 Reviewer)
  • Description
    • Concise: Keep it short and simple
    • Add unit tested evidence
      • video attachment or
      • images (postman if API changes only)
    • Add Jira ticket
  • CI pipeline succeed
    • Successful Installation
    • 0 Lint Issues
    • = 80% Test Coverage

  • No Merge Conflicts
  • Commits shouldn't diverge
  • Post Merge (Jenkins Integration)
    • Lower Dev-like Environment: Auto Deployed
    • Higher Prod-like Environment: Manual Button Click by Maintainers / Owners

Email spoofing prevention

DMARC setup

  • Visit https://mxtoolbox.com/DMARCRecordGenerator.aspx
  • Generate DMARC record
  • Visit dashboard where domain is purchased
  • Visit DNS settings
  • Add a record with below details
    • Type: TXT
    • Name: _dmarc
    • Value: v=DMARC1; p=none; rua=mailto:; pct=10;
    • Proxy: DNS
    • TTL: Auto

DKIM setup

  • Visit https://dmarcly.com/tools/dkim-record-generator
  • Generate DKIM record
  • Copy value
  • Visit dashboard where domain is purchased
  • Visit DNS settings
  • Add a record with below details
    • Type: TXT
    • Name: ._domainkey
    • Value:
    • Proxy: DNS
    • TTL: Auto
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment