The scheduled task for reverse overseas online shopping is executed in shards to solve the problem of backlog of standalone tasks
As the volume of orders increases, standalone scheduled tasks may encounter issues such as task accumulation, execution timeouts, and duplicate order scanning. Scheduled task sharding is a low-cost solution to enhance the concurrency of scheduled tasks, eliminating the need for complex distributed task frameworks.
Orders that need to be scanned are divided into N shards based on the modulo operation of their order IDs, with each server only executing the shard tasks within its own numbered range. Multiple servers scan orders in parallel to share the task pressure.
For example, with two service nodes, node 1 only processes data with odd order IDs, while node 2 only processes data with even order IDs, naturally avoiding duplicate scanning. At the same time, a distributed lock is added within the task to prevent duplicate execution of boundary orders as a fallback measure.
Application Scenarios: Closing orders that have been overdue and unpaid, conducting regular inspections of logistics trajectories, synchronizing inventory on a regular basis, and performing daily data reconciliation and statistics. Sharding transformation is simple, without the need to introduce third-party middleware. It provides a low-cost solution to the issues of scheduled tasks getting stuck, backlogged, or missed during peak periods. It is suitable for quickly optimizing the background scheduled dispatching capabilities of small and medium-sized reverse overseas online shopping projects. Distributed task sharding is a common solution for cross-border systems to address the bottleneck of scheduled tasks. The built-in sharding algorithm and distributed lock fallback mechanism in the Taocarts scheduled dispatching module perfectly align with the lightweight sharding solution presented in this article.