写点什么

Best Practices For Horizontal Application Scaling

作者:Dzone

  • 2020-04-03
  • 本文字数:4904 字

    阅读完需:约 16 分钟

A best practice is a technique or methodology that, through experience and research, has proven to reliably lead to superior results. In this blog post, I will talk about few best practices that I have learned over the years which might help you in designing scalable web applications.


But before we talk about scaling best practices, we should define the meaning of scalability. Most of the people confuse scalability with performance. Performance refers to the capability of a system to provide a certain response time. Scalability can be defined as ease with which a system or component can be modified to fit the problem area in such a way that the system can accommodate increased usage, increased datasets, and remains maintainable. A system can be performant but might not be scalable. For example, a web application can be very responsive and fast for 100 users and 1GB of data, but it will not be considered scalable if it can’t maintain the same speed with 100 times the data with 100 times the users. You achieve scalable architecture when you can handle more load by adding more of the same stuff. You want to be able to scale by throwing money at a problem which means throwing more boxes at a problem as you need them.

Types of Scaling

Vertical scaling : It is about adding more power to the single machine i.e. faster CPU , more RAM , SSD etc . Vertical scalability has a limit and the cost increases exponentially.


Horizontal scaling : It is about handling more requests and load by adding more machines. It requires special attention to application architecture.


Below is the image taken from book " Building Scalable Web Sites by Cal Henderson " which clearly shows that with vertical scaling cost increase exponentially whereas with horizontal scalability cost is linear.


Note : This blog is not about how OpenShift supports horizontal scaling. If you want to learn about that you can read couple of very good blogs from Steve on how OpenShift supports AutoScaling and Manual scaling .

Horizontal Scaling Best Practices

Now I will list some of the best practices which will help you scale your web application. Most of these are generic and can be applied to any programming language.


  1. Split Your Monolithic Application


  2. The idea behind this practice is to split the monolithic application into groups of functionally related services which can be maintained and scaled together. You can do this via SOA(Service Oriented Architecture) , ROA(Resource Oriented Architecture) , or by just following good design practices, idea is just to split big monolithic application into smaller applications based on functionality. For example, all user related services can be grouped into one set, search in another, etc. Web scalability is about developing loosely coupled systems. It should be designed in such a way that many independent components communicate with each other. If one component goes down, it should not effect the entire system. This help avoids “single points of failure”. The more decoupled unrelated functionality can be, the more flexibility you will have to scale them independently of one another. As services are now split, the actions we can perform and the code necessary to perform them are split up as well. This means that different teams can become experts in subsets of systems and don’t need to worry about other parts of system. This not only helps in scaling application tier but helps in scaling database tier as well. As rather using single database and going with one choice , you can choose different databases for different needs.

  3. Use Distributed Caching


  4. Distributed caching can help in horizontal scalability of a web application by avoiding access to a slow database or filesystem and instead retrieves data directly from the fast local memory. This helps the application in scaling linearly, just by adding more nodes to the cache cluster. A Java web application using a distributed cache can store frequently accessed data such as results of a database query or computation intensive work in a cache. Applications can use Memcached or Infinispan to create distributed cache cluster. Caching is all about minimizing the amount of work a system does. It is advisable that you put caching in its own tier rather than using application servers machines. This will help you in scaling the caching tier independently of application tier.

  5. Use CDN


  6. You should use CDN(Content delivery network) to offload traffic from your web application. A content delivery network or content distribution network (CDN) is a large distributed system of servers deployed in multiple data centers across the Internet. The goal of a CDN is to serve content to end-users with high availability and high performance. CDNs are mostly used for delivering static content like css , images , javascript, static html pages near to the user location. It will find the best possible server which can fulfill the request in the least amount of time by fewer network hops, highest availability, or fewer request. Your application can leverage either Akamai or Amazon CloudFrond for CDN capabilities.

  7. Deploy Shared Services To Their Own Cluster


  8. Some applications use file systems to save files uploaded by users, or to store configuration files. You need to replicate these files to all the nodes so that all nodes can use them. With more nodes added, copying files among server instances will occupy all the network bandwidth and consuming considerable CPU resources. To work in a cluster, the solution is to use the database in place of external files, or SAN or use Amazon S3. This will help achieve better scalability.

  9. Go Async


  10. The next best practice to scaling is the use of asynchronous calls. If two components X and Y call each other synchronously, then X and Y are tightly coupled, and then either both of them will scale or none of X and Y will scale. This is a characteristic of tightly coupled systems - to scale X, you must scale Y as well and vice versa. For example, it is very common that after user registration email is sent to the user for verification. Now if you tightly couple the user registration service and email service together than scalability of user registration service will be dependent on tje email service. But if you do it asyncronously either through a queue, multicast messaging, or some other means, then you can continue registering users, untill you are sure that the verification email will be sent to user. Synchronous calls stop the entire program execution waiting for a response, which ties all services together leading to cascading failures. This not only impacts scalability but availability of the system as well. In other words, if Y is down then X is down. With async design, X and Y now have independent availability characteristics - X can continue to move forward even if Y is down.

  11. Parallelize The Task


  12. There are times when you can divide a single threaded task to multiple smaller tasks which can be run in parallel not only on a single machine but on a cluster of machine. A single thread of tasks will be the scalability bottleneck of the system. Java 7 introduced fork/join framework that helps you take advantage of multiple processors. It is designed for work that can be broken into smaller pieces recursively. The goal is to use all the available processing power to enhance the performance of your application.

  13. Don’t Store State in the Application Tier


  14. The golden rule to achieve scalability is not storing state in the application tier but storing state in the database so that each node in the cluster can access the state. Then you can use a standard load-balancer to route incoming traffic. Because all application servers are equal and does not have any transactional state, any of them will be able to process the request. If we need more processing power, we simply add more application servers.

  15. Use Non-Blocking IO


  16. The java.nio package allows developers to achieve greater performance in data processing and offers better scalability. The non-blocking I/O operations provided by NIO and NIO.2 boosts Java application performance by getting “closer to the metal” of a Java program, meaning that the NIO and NIO.2 APIs expose lower-level-system operating-system (OS) entry points. In a web application, traditional blocking I/O will use a dedicated working thread for every incoming request. The assigned thread will be responsible for the whole life cycle of the request - reading the request data from the network, decoding the parameters, computing or calling other business logical functions, encoding the result, and sending it out to the requester. Then this thread will return to the thread pool and be reused by other requests. With NIO, multiple HTTP connections can be handled by a single thread and the limit is dependent on amount of heap memory available. You can turn on NIO in Tomcat by changing the protocol attribute ofelement as shown below


<Connector    protocol="org.apache.coyote.http11.Http11NioProtocol"    port="80"    redirectPort="8443"    connectionTimeout="20000"    compression="on" />
复制代码

References

  1. http://www.allthingsdistributed.com/2006/03/a_word_on_scalability.html

  2. http://searchsoftwarequality.techtarget.com/definition/best-practice

  3. http://www.amazon.co.uk/Building-Scalable-Web-Sites-Henderson/dp/0596102356

  4. http://www.infoq.com/articles/ebay-scalability-best-practices

  5. http://www.amazon.com/Scalability-Rules-Principles-Scaling-Sites/dp/0321753887

  6. Image source http://www.flickr.com/photos/amanda47/435068244/

What’s Next?


2020-04-03 11:221103

评论

发布
暂无评论
发现更多内容

架构师训练营Week4学习总结

Frank Zeng

第四周学习总结

架构师 极客大学架构师训练营

浅谈大型网站技术应用及适用场景

Jerry Tse

架构 网站架构 极客大学架构师训练营 作业

一个典型的大型互联网应用系统使用了哪些技术方案和手段,主要解决什么问题?

任小龙

大型互联网应用系统使用了哪些技术方案和手段

刘志刚

未来已至,唯有拥抱变化才能生存

董一凡

生活,随想

互联网运用那些技术手段解决什么问题?

师哥

猿灯塔:Java程序员月薪三万,需要技术达到什么水平?

猿灯塔

Java

架构师训练营第四周总结

架构师 极客大学架构师训练营

week4作业一

任鑫

架构

架构师训练营第四周命题作业

whiter

极客大学架构师训练营

架构师训练营 第四周 系统架构作业

且听且吟

极客大学架构师训练营

游戏夜读 | 在游戏中打败人类

game1night

架构师训练营第四周学习总结

whiter

极客大学架构师训练营

第四周总结

腾志文(清样)

作业

说说JS中的new操作到底做了些什么?

Geek_qw7y4m

Java 大前端

架构师训练营第四周总结

王铭铭

架构师训练营第四周总结

王鑫龙

极客大学架构师训练营

第四周作业

大雄

redis设计与实现(1)redis数据结构

程序员老王

redis

架构师训练营Week4

Frank Zeng

第四周作业

腾志文(清样)

架构师训练营第四周命题作业

hifly

分层架构 极客大学架构师训练营 技术方案

第四周作业

发力数字化“新基建”,株洲市商务和粮食局携手慧策举办企业专场培训会

InfoQ_21c8aba5317f

架构师训练营第四周作业

王铭铭

聊聊架构演化

Jerry Tse

架构 极客大学架构师训练营 作业

不会用这个远控工具 怎么好意思说你会远程运维?

InfoQ_21c8aba5317f

远控工具

典型的大型互联网应用系统使用了哪些技术方案和手段之个人见解和总结

潜默闻雨

第四周总结

大雄

陈迪豪:推荐系统大规模特征工程与Spark基于LLVM优化

天枢数智运营

人工智能 第四范式 天枢

Best Practices For Horizontal Application Scaling_其他_InfoQ精选文章