Check How Mcclatchy Interactive Employs Open Source Search


Highlights Staying competitive is the name of the newspaper game The third-largest newspaper publisher in the U.S., McClatchy Company owns 30 daily newspapers in 29 markets across the country. This includes the Sacramento Bee, its first paper and one of the oldest in the West, the Miami Herald, The News and Observer, and others. The unforgiving economic climate has hit the industry hard, accompanied as it is with fundamental shifts in reader and advertiser behaviors. As a market leader, McClatchy is vitally focused on their online business model. McClatchy Interactive has more experience than most they launched one of the first electronic news sites on the Web, The Nando Times, in the Spring of 1994. Today, McClatchy's mission is to establish its local papers as the leading source of online news within their respective markets, with compelling local information online, along with comprehensive news, advertising, e-commerce and other services.

Helping online readers find the relevant information they want is a cornerstone of their online initiative. Open source technologies contribute directly to McClatchy's business model by minimizing the both capital and operational costs without compromise in customer experience. McClatchy Interactive chose the leading open source search solution, Solr, to search classified ads across all online properties. Solr has already helped McClatchy Interactive properties reduce costs dramatically, while improving on the same high quality of service to its online readers. "Solr has done wonders for us. It is easy to understand and deploy, and reduced our costs drastically," according to Doug Steigerwald, software developer at McClatchy Interactive. Making the leap from proprietary system to open source search The company's legacy search system was built on a proprietary, pre-packaged ="_blank"> enterprise search software solution that was increasingly expensive to maintain and modify while addressing new business requirements. In 2007, the company decided to bring in-house its Web site search functionality, and build a search service that could handle classified ads across all McClatchy properties.

Primary responsibility for development fell to Doug, although he had some part-time assistance from another member of the software development group. There were two requirements. The first was that the new search system had to have the same features and capabilities as the existing search product. Second, the solution had to be seamless, transparent to support personnel and the McClatchy newspapers. "It had to be absolutely no work for anybody other than development," said Doug. "And it had perform the same or better as what we already had." Why Solr was chosen Solr is the leading open source search solution, and features high performance and comprehensive capabilities. These include faceting the ability to drill down on characteristics of the desired result. For example, when looking for a used car, a reader may want to limit search results by price range or model type, and then drill down to find only those cars that are nearby. Any search solution would need to enforce McClatchy's search policy requirements. Stories which can be searchable or non-searchable appear in many places across McClatchy Interactive properties. A searchable story must remain so, even if it appears in a non-searchable section. The converse is also true, controlling which results are excluded from a search. Before a final decision was made, extensive comparisons of Solr and the existing search package were undertaken, in which Solr delivered better, more relevant results. The developers also checked to make sure that Solr's search processes stored data in the same way as the legacy system, so no problems would arise with other related applications after the new solution was rolled out to production.

McClatchy's Solr implementation was able to replace the legacy software without changing any significant back end functionality. Doug says, "After researching it, we felt Solr could do everything we'd done with our existing proprietary system." Real, fast results The project began in September 2007 and was in production by April 2008. With experience in open source development, the developers began evaluating and building a search system using Solr. Basic development, including the training classes for the two developers, was completed in January 2008. Once the pilot implementation was up and running, efforts moved to improving speed and accuracy. Two months of QA ensured everything would work as intended. In order to meet their targets for both performance and relevance, McClatchy engaged Lucid Imagination consulting services to help them get the most out of their implementation as soon as possible. The team put together a presentation about what McClatchy was doing and asked for an expert opinion on how the system could do better in terms of performance. Lucid worked with McClatchy to carry out extensive analysis of logs and production for all sites, to profile the volume, diversity, and frequency of data being indexed. For McClatchy, Lucid's recommendations focused largely on configuration and schema for example, what field options could be closed or disregarded to increase indexing and query response time. Doug says, "Lucid Consulting was definitely worth it. They helped things move along more quickly by pointing out ways to better tune Solr, based on our usage. We could have spent a couple of weeks benchmarking and refining our implementation, but Lucid Consulting was able to optimize our specific search environment in a day or two." McClatchy Interactive today The Solr implementation achieved impressive cost savings and performance. Because there are no licensing costs associated with Solr, McClatchy enjoyed significant savings by eliminating the licensing fees associated with the previous packaged software application they had purchased. In addition, one of the most impressive benefits of this Solr implementation was the dramatic reduction in hardware requirements. The old search required 24 production servers, while McClatchy's Solr search service runs on just three of the existing servers. The remaining 21 servers have been freed up for other uses. "We really reduced the number of servers we needed," states Doug. "We may add one or two more in the future to handle extra traffic and roll out some new services." All 30 McClatchy dailies are running on the new system, and the combined number of classified and news searches reach 700,000 per day primarily from classified queries.

Every ad category and classification is stored within a Solr index. Solr also hooks into the company's content management system, which serves all its newspapers, to index other searchable data such as stories, movie show times, and so on. In addition, the entire front page of online properties is powered by Solr, including news search, movie search, and more. Partly because it was so easy to implement, Doug recommends Solr to other developers who are just beginning to explore open source. "Solr is free and has a huge, active user base, which is very helpful," says Doug. "It was a different experience than the proprietary software we were using. We found it was easier to get support by going open source. The Solr community provides support through the developer e-mail list, where you can find out how others have already solved problems you might have." McClatchy Interactive is evaluating Solr to take advantage of other capabilities. For example, Doug is evaluating Solr's More Like This functionality, to help readers find related stories and increase site stickiness. And as Doug points out, there's no extra charge for this functionality.

Data Center Consolidation


Data Center Consolidation One of the major initiatives confronting many businesses today is data center consolidation. Data center consolidation saves energy, reduces physical space requirements, and optimizes the processing of information, all of which have the added bonus of reducing carbon footprints andhelping protect the environment. Security costs play a central role in data center consolidation, accounting for about 10 percent of the total infrastructure costs. Conventional methods of implementing network security and arranging the physical space of the data centers, such as stacking property boxes one on top of the other, is outdated and inefficient. All of this equipment increases capital costs, operating expenses and energy consumption. Milli Micro Systems has a proven approach for the consolidation of the security infrastructure. Our solutions have been implemented in some of the largest networks on the planet, including US government agencies. By moving the physical network and security devices to a highly scalable, highly redundant platform or virtual environment, MMS helps customers consolidate devices, as well as the reduce capital and operations costs.

Our SolutionsMilli Micro Systems creates consolidated data center intensive virtualization technologies are featured in public sector organizations and companies across the US. These solutions create sustainable environments that are more profitable and easier to experts can help you examine your current working practices and user demands, and will then audit your systems and develop an effective strategy for data center consolidation. Our services include designing, planning and executing your data center consolidation, including all aspects of network architecture, connectivity, integration and migration, as well as ongoing support and managed services. The MMS Approach to Data Center ConsolidationMMS can help you build effective solutions that incorporate all the networks, user services and applications to one or more centralized locations, which lowers operating costs and the costs of future investments. These improvements also fortify the safety and security of your organization, and create reliable platforms that can be used for collaborative applications, such as Unified Communications.Using solutions based on global data services, enterprises can accelerate their applications and ensure effective consolidation of global and regional data centers. MMS solutions significantly improve the performance of critical applications running over the WAN, and at the same time reduce the use of network bandwidth on the WAN. This increases employee productivity and makes it possible for IT managers to further optimize IT infrastructure and reduce IT maintenance course, organizations may still need a backup center for disaster recovery.

MMS can help organizations improve the speed of backup and data replication for all data centers. MMS Data Center PlatformWith MMS's platform, you can:??? Establish a availability and performance benchmarks??? Track the success of the data center consolidation strategy??? Compare the performance of the application before and after consolidation??? Effectively manage complex environments with multiple data centers??? Ensure that web applications are optimized for display on all strategic browsers, operating systems and mobile devices (before and after consolidation)??? Ensure that specialized environments function as before, during and after the consolidation??? Provide complete visibility throughout the distribution chain before, during and after the transformation to a virtualized data center??? Provide full visibility regarding the use, bandwidth and containment applications in all aspects of your network to include links to Internet, LAN and WAN??? Identify targets for application performance before they are changed by using the response time indicator in order to establish realistic and achievable service level objectives??? Centralize the entire inventory and use performance monitoring tools in a comprehensive management solution