Most of the data we store is unstructured

[rakhirhif8963]

Data silos continue to be a challenge for data engineers. But 2024 will bring a glimmer of hope as a centralized form of data orchestration comes to the fore, according to Molly Presley, senior vice president of marketing at Hammerspace. “Organizations will begin to move away from the ‘store and copy’ world and into the world of data orchestration,” she says. “Thanks to advances in AI, we now have robust tools to analyze data and extract actionable insights. However, file storage infrastructure has not kept up with these advances. Unlike solutions that try to manage silos and distributed environments by moving copies of files from one location to another, data orchestration helps organizations integrate data into a single namespace from different silos and locations and automate the placement of data where and when it’s most valuable, making it easier to analyze and extract insights.”

As it accumulates, it becomes a real problem, but 2024 will bring new ways to manage it, says Anand Babu Periasamy, co-founder and CEO of MinIO. “In 2024, we will see an explosion of truly unstructured data (audio, video, meeting recordings, conversations, presentations) as AI applications come online. This is very useful content for training AI, and collecting it into an AI data lake will greatly improve the intelligence of the enterprise as a whole, but it will also create unique challenges,” he says. “There are specific performance challenges when dealing with tens of petabytes that typically cannot be addressed with traditional SAN/NAS solutions — they require the attributes of modern, high-performance object storage. This is why most AI/ML technologies (e.g., OpenAI, Anthropic, Kubeflow) use object storage, and this is why most databases are moving to object storage.”

According to Forrester, the amount of argentina mobile database data managed by enterprises will double in 2024, opening up new, potentially lucrative opportunities for AI. “Data and analytics decision makers around the world say that only 27% of the data their organizations manage is unstructured,” the research group notes. “Generative AI will double that number as companies implement more conversational experiences for customers and employees. Enterprises will struggle to store, analyze, and make sense of this flood of unstructured data. This trend will manifest itself in the area of ​​data pipelines, where 80% of new pipelines built in 2024 will be dedicated to collecting, processing, and storing unstructured data.”