9 Replies Latest reply: Nov 10, 2014 2:38 PM by DarkData RSS

Ask the Expert: Redefining Video Surveillance Storage from Edge to Core

Roberto Araujo
YOU MAY ALSO BE INTERESTED ON THESE ATE EVENTS...

Ask the Expert: Redefining Entry Level Storage

Ask the Expert: Rethink, Redefine and revolutionize your data center on your journey to the 3rd platform.

Ask the Expert: Installation and configuration of EMC storage arrays solution pack

Welcome to the EMC Support Community Ask the Expert conversation. EMC recently announced purpose-built storage solutions for the video surveillance market including the VNX-VSS100 and new partnerships (Distributors/VMS). Our Experts have several decades worth of experience in the Video Surveillance market and are here to answer any and all questions you may have related to the market or the new EMC solutions announced on September 30th. If you missed the live announcement, view it here, and then ask your questions below!


Your Hosts:

profile-image-display.jspa?imageID=11778&size=350

Frank McCarthy is Director of Video Surveillance Solutions within the Global Corporate Practice at EMC. Frank joined EMC in 1998 and in 2004 established the Engineering and Solution development team as part of the Global EMC Solutions Group where he has been responsible for all facets of solution development, benchmarking, and amalgamation of EMC’s framework of software and hardware products enabled by best in breed partner technologies in the security industry. He has 23 years of experience as an IT professional with a focus on security solutions since 2004 and is a seasoned security technology innovator, working with both large and small companies. Frank has held positions with various Fortune 100 companies as a Systems Integrator/Analyst/Engineer, Technical Account Manager, Service Account Manager, and Sales Engineering Manager.

profile-image-display.jspa?imageID=8412&size=350

Bryan Berezdivin is the chief architect for the Emerging Technologies Division of EMC, focused on application integration and optimizations with surveillance, ISR, Geospatial, and analytics applications. In this role, Bryan works with most of the leading software, camera, geospatial, and analytics vendors to derive optimal architectures for Internet scale problems such as video surveillance as a service offerings, drone surveillance, and city-wide surveillance systems. Bryan recently ran the high performance computing (HPC) solutions at NetApp and before that was lead architect for Cisco’s physical security software in the Federal market. Previously engaged in over 70 projects including large government, Defense, healthcare, airport, casino, and commercial surveillance systems, NOAA’s NPOESS system, as well as common operating picture (COP) software for rotary and fixed wing aircraft.

profile-image-display.jspa?imageID=9243&size=350

Ken Mills has been a leader in the Video Surveillance industry for over 15 years. Ken Mills joined EMC2 in late 2013 to once again help build another thriving business around Surveillance. Ken is responsible for the Global Go To Market, Channel, Advertising, and Sales Enablement of EMC2 for the Surveillance and Reconnaissance business. Ken watched his father build a successful business in the Surveillance industry and after time in the Navy as a Nuclear Engineer found his back to the world of Surveillance. Ken was also a founding member of the Cisco Systems incubation organization called Emerging Technologies and spent almost 8 years building a thriving business for Cisco focused on Surveillance, Access Control, and Emergency Response.

profile-image-display.jspa?imageID=11672&size=350

Joe Catalanotti is a Product Marketing Manager with EMC Enterprise Midrange Systems Division focused on VNX products and solutions. Joe has over 25 years of product and channel marketing experience in storage hardware/software, asset management, and CAD technology. Joe has been instrumental in the development and execution of go-to-market plans, product launches, and other facets of product marketing. Joe holds a BS degree in Industrial Engineering and Management (Sigma Epsilon Rho) from Northeastern University as well as a degree in Architecture from Wentworth Institute f Technology.

Event date and duration: October 13 - 27.


Share this event on Twitter:

>> Join our Ask the Expert: Redefining Video Surveillance Storage from Edge to Core http://bit.ly/1B3O6om #EMCATE <<

  • 1. Re: Ask the Expert: Redefining Video Surveillance Storage from Edge to Core
    Roberto Araujo

    This discussion is now open for questions. We look forward to a lively and informative event.

     

    Best regards,

    Roberto

  • 2. Re: Ask the Expert: Redefining Video Surveillance Storage from Edge to Core
    Niki Vecsei

    Hi VSS Experts,

     

    You spent a lot of time with customers and EMC partners at ASIS doing live demos. Tell us about it!


    • What were to most common customer questions about the new Video Surveillance Solutions launched by EMC?
    • What pain points are the new solutions addressing that matter the most to EMC customers?

    VSS live demo at ASIS.JPG.jpg

  • 3. Re: Ask the Expert: Redefining Video Surveillance Storage from Edge to Core
    DarkData

    One of the more common and interesting questions was how/why to use the EMC solutions to enable analytics in line with the security goals of the various folks we spoke to.   The main goal by these security personnel and decision makers was to help them handle unpredictable scenarios and allow them to utilize their surveillance system data to help without going kookoo in the process ;-)

     

    The most compelling discussion was with a stadium customer.  They want to allow their security personnel identify where a particular "threat" has visited to evaluate potential danger during, before, or after an event.  Most video analytics systems are setup to accommodate identifying real time threats by specifying an activity/scenario to look for, which is all predefined.  The issue with these systems is they do not help entirely with the use case defined, which requires identifying a new person to look for across your site. The idea is to reduce the last 2-4 days of video across 100s of video cameras in the stadium and surrounding area to a few minutes of relevant video for 1-2 security analysts/personnel to evaluate and decide there is no threat or perhaps act accordingly on the potential threat.  In the example provided, this might be via pattern matching of a "face/body/hat/shirt color" to the video over the last few days.  We refer to this as post processing and is common in many systems that EMC is deployed outside of surveillance. 

     

    One element of the post processing architecture that is important for such an analytics ready system is to have 1) shared storage of the video data/metadata and 2) standards based protocols for these analytics applications to access data.   EMC refers to this approach as a Data Lake and can reviewed in the generic IT context in the following reference:  Isilon scale-out Data Lake (white paper)

     

    By depicting a system with an Isilon video repository, evidence repository, case management repository, and even distributed sensor repository, the analytics applications could use any file (SMB, NFS, FTP), object (SWIFT, HTTP), or even Analytics (HDFS) protocol to access and process this data to make the security personnel's life and mission easier.  The key here is that the system simplicity also avoids any IO bottlenecks due to silo's of data. 

     

    Hope this is helpful,

    Bryan

  • 5. Re: Ask the Expert: Redefining Video Surveillance Storage from Edge to Core
    christiegifrin

    During the live Q&A session at the launch there was one concern that kept on coming up, and that is how the migrations from the edge to the core should be tackled.  What are the best mechanisms and preparations and what effects such a transfer may cause to the bandwidth? Can you give some detailed information on these  concerns?  Thanks in advance!

  • 6. Re: Ask the Expert: Redefining Video Surveillance Storage from Edge to Core
    mccarf

    The Video Surveillance Development team is currently assessing software based, third-party applications with key industry partners that can be deployed with the Video Management Software (VMS) at the edge to facilitate the data transfer. The applications are highly efficient and secure providing an accelerated data transfer capability that is designed to meet the demands of the video surveillance dataset.  The initial approach is to provide enhanced file transfer efficiency for manually moving events of interest to the core.  Eventually, we plan to augment the file transfer efficiency of VMS applications that have built-in file/event transfer features. 

     

    The video surveillance dataset presents some challenging aspects that a file transfer service has to take into account.  The dataset is comprised of both highly compressed video and in some cases metadata that describes or defines the specific attributes or event(s) associated with the video.  The video segment is by far the largest and most bandwidth and CPU intensive part of the file transfer from edge to core.  Since video is highly compressed, hardware based WAN optimizations that utilize compression is of little benefit in minimizing the impact on bandwidth.  We also see large files, >100MB, as a typical file size so the file transfer service should be optimized for large file transport and support sub-file or segmented file transfer to allow for checkpoint restarts in the event of an interrupted transfer session. 

     

    These challenges are best met by a software based approach that can dynamically manage the available bandwidth via flow and congestion control, reduce network protocol overhead through selective retransmission, and provide centralized management that can centralize data from hundreds or thousands of edge sites to a core site.

     

    We plan to wrap up our assessment phase this quarter and will have more details regarding this aspect of the edge to core architecture in that timeframe.

     

    Frank

  • 7. Re: Ask the Expert: Redefining Video Surveillance Storage from Edge to Core
    Ali_N

    Hello All,

     

    As far as I know , video surveillance system tends to be large in IO Size with a very high percent  of random distribution, (because of VMS) and they are supposed to write much more than read. At least I have analyze storage system of a customer and found out their average IO Size was 120KB and distribution was 98% write & only 2 % read.

    So, Why does everyone use NL-SAS (7.2K rpm) for data storing of video surveillance system?  VSS-100 is equipped with NL-SAS as well.

    SAS has a higher IOPS, and in an environment with large IO Size, they can gain more bandwidth.

    So Is there any technical reasons for choosing NL-SAS in VSS or it is only because of  lower price and higher capacity?

     

    Thanks,

    Ali

  • 8. Re: Ask the Expert: Redefining Video Surveillance Storage from Edge to Core
    joecat

    Video surveillance storage at the Edge is characterized by many attributes/requirements including availability, simplicity, and certifications with VMS. Another important attribute is cost. We've configured the VNX-VSS to reflect as many of these Edge requirments as possible including scalability and price understanding that the VNX-VSS also has innate capabilities to deliver high system bandwidth (which we've tested) of up to 500 MB/sec - combined with the large capacity drives - gives customers a leading "edge" solution to support a few hundred cameras at multiple bit rates. 

  • 9. Re: Ask the Expert: Redefining Video Surveillance Storage from Edge to Core
    DarkData

    The size of video writes for the majority of Windows based VMS vendors comes in 1MB blocks for variety of file sizes.  The other writes present on the workload vary per VMS vendor implementation, but can be audio/metadata file writes.  The Linux based VMS vendors (maybe 10% of market), are all a little different.  Remember to keep an eye on max and min write sizes that the average is being sampled from and you might be able to distinguish writes for the video files versus writes for other parts of the data recorded.

     

    Video surveillance, in my humble opinion has a few workloads to be aware of and you can treat differently if desired.  The most storage centric is video archiving, which typically is highly sequential and nice sized blocks.  The other spectrum of this is the indexing, which is usally on a database, and this is higly transactional and small block sizes.  Then there are foundational OS functions associated with the normalization of the video stream (usually in memory).

     

    What is interesting is I 100% agree with your assessment of video surveillance workload on block based systems, but I disagree that this is true for non-block based systems.  By using a scale-out filesystem, use of pooled resources (disks, memory, CPU, network) the ability to use SATA disks for video archiving workload is well within bounds of handling the workloads by the storage subsystem if it uses a scale-out filesystem.  With EMC Isilon, we are actually able to use our lowest end node types and disks for video surveillance and achieve bandwidths required for the systems with little/no issues even during an entire Node of 36 drives is in failure state (worst case scenario testing).  The issue in the market, actually, is that the memory allocation mechanisms of the software systems needs improvement many times to go above and beyond in the throughputs.  This is because file/object interfaces add latency when compared to SCSI interface due to protocol semantics and the networks.  EMC has worked with quite a few VMS providers on this front and have very good solutions to leverage.

     

    Sorry about diatribe, but it is a fun space!!!

     

    Bryan