It’s no wonder that a SAP’s HANA in-memory database has caught the attention of many in the transactional, BI and Big Data worlds. HANA uses memory as its primary medium, with disk being used merely for redundancy. While there’s of course a limit to how much memory you can have on a single server, HANA employs a scale-out architecture that lets you expand the database beyond the single-server boundary. And the memory across all nodes in the cluster is still usable as a single pool. If terms like node, cluster and scale-out make you think of Big Data, that’s a reasonable reaction. HANA’s most emblematic workloads are in the Big Data space. Sentimental Big Data TDWI World Conference in San Diego, SAP announced general availability of the its rapid-deployment solution for sentiment intelligence with SAP HANA. The solution hooks up HANA to Twitter, Facebook or any Web data source with a public API. It then puts text mining … [Read more...] about SAP HANA does Big Data…with ERP, CRM and BI savvy
What does big data mean
What does big data mean for cloud providers? This is the topic under discussion at a EuroCloud UK meeting next week. This blog post is both a reflection on the topic as well as a taster for the event itself, which has an interesting and knowledgeable line-up of speakers — more details on that below [disclosure: I'm chair of EuroCloud UK, so I do have a vested interest in the success of the event]. I've seen a lot of definitions of big data over the past couple of years. I prefer a fairly loose interpretation: when it comes to industry buzzwords, my policy is to 'go with the flow' of popular usage, irrespective of what purists may argue. The underlying trends are what really matter, and the reason this particular buzzword has become commonplace is the confluence of cloud computing with smart devices: As well as providing on-demand access to a wealth of private and public data resources, the cloud delivers inexpensive storage capacity and elastically scalable, pooled computing … [Read more...] about Cloud providers working with big data
Faced with the ongoing confusion over the term 'Big Data,' here's a handy - and somewhat cynical - guide to some of the key definitions that you might see out there. (1) Big Data as Exponential Data Growth Despite what Wikipedia says, most people in the industry generally agree that Big Data isn't just about having more data, but that was indeed the term's first meaning in the late 1990s (even though warnings about the exponential rise of data volumes go back until at least the 1940s). (2) Big Data as Data Characteristics Big Data as the three Vs: Volume, Velocity, and Variety. This is the most well-known definition, first coined by Doug Laney of Gartner over twelve years ago. Since then, many others have tried to take it to 11 with additional Vs including Validity, Veracity, Value, and Visibility. (3) Big Data as New Technology Why did Big Data suddenly become such a widely used term? It wasn't simply because we do indeed now have a lot more volume, velocity, and variety than a decade … [Read more...] about What Does “Big Data” Mean? A Cynic’s Guide
My UC Berkeley programming students often ask me which programming language is the most popular. It's a simple question with a far-from-simple answer. Let's start with why people want to know. Usually, at least when it comes to students, they ask which language is the most popular because they want to learn subjects where there are jobs. For most people, it makes very little sense to enter a field with knowledge about something that's not in demand. special feature How To Make IT More Agile Business executives are increasingly moving to an IT environment that is no longer focused on big, long projects but shorter, more sustainable efforts to drive change and innovation. Here are the pros and cons, and how to make it happen. Read More Other times, programmers who are already skilled want to get a feel for whether their current skills are relevant, or whether it's time to look at other languages because shifts in popularity might mean it's time to bone up on a new language. … [Read more...] about Which programming languages are most popular (and what does that even mean)?
If you’re as old as I am, you’ve seen a lot of different names for the idea of using data and analysis to better manage organizations. First it was called decision support systems (DSS); then executive information systems (EIS); then online analytical processing (OLAP); then business intelligence (BI). It now appears that yet another term for this concept is growing in popularity. The new label is “business analytics.”Is “business analytics” the new name for “analytics,” or for “business intelligence” more broadly? Is it a good name? What does it mean?In the past, business intelligence was an umbrella term that included both reporting on data and analyzing it with statistical and other quantitative approaches, although reporting was certainly the bulk of the activity. Hence the more recent need for the term “analytics,” which was more focused on quantitative analysis. I’m getting the feeling, however, that … [Read more...] about What Does “Business Analytics” Mean?
As part of the Federal Government's Big Data Extravaganza, the Defense Advanced Research Projects Agency (DARPA) detailed some of its programs the agency says could benefit from the $200 million research program announced today by President Obama. The Big Data Research and Development Initiative is expected to bolster the tools and techniques needed to access, organize, and glean discoveries from huge volumes of digital data. In other news: 14 cool, but off-beat inventionsDARPA's big data programs include: The Anomaly Detection at Multiple Scales (ADAMS) program looks at the problem of anomaly-detection and characterization in massive data sets. In this context, anomalies in data are intended to cue collection of additional, actionable information in a wide variety of real-world contexts. The initial ADAMS application domain is insider-threat detection, in which anomalous actions by an individual are detected against a background of routine network activity. The Cyber-Insider Threat … [Read more...] about DARPA does Big Data in a big way
As companies embrace big data, they're in the market for high-level strategists and communicators. Do you have the chops to snag a big data job?As big data gathers momentum, it's helping to create big career opportunities for IT professionals -- if they have the right qualifications.According to a report published in 2011 by McKinsey & Co., the U.S. could face a shortage by 2018 of 140,000 to 190,000 people with "deep analytical talent" and of 1.5 million people capable of analyzing data in ways that enable business decisions.Companies are, and will continue to be, looking for employees with a complex set of skills to tap big data's promise of competitive advantage, market watchers say. "There's no question that the No. 1 requirement [for] enterprises that are serious about gaining a competitive advantage using data and analytics is going to be the talent to run that program," says Jack Phillips, CEO of the International Institute for Analytics (IIA), a research firm.But what … [Read more...] about Big data means big IT job opportunities — for the right people
At last week's SVForum Big Data Event, you could have been forgiven for thinking that it would be all Hadoop, all the time. Even though Hadoop was featured prominently, it was by no means the only topic.It's no secret that Hadoop and big data are, well, big. The explosion of new data sources such as social media and website clickstreams offers new types of information and, potentially, the opportunity to generate new insights. Hadoop has been front and center in this trend, offering the MapReduce method across distributed data collections to make it possible to analyze far larger amounts of data than traditional approaches can manage. Gartner: Hadoop Will Be in Two-Thirds of Advanced Analytics Products By 2015However, it's obvious that Hadoop, despite its obvious benefits, is not a business intelligence magic bullet. A number of speakers noted the shortcomings of its batch nature; it can take hours to get return results that, once examined, are clearly not what was desired, thus … [Read more...] about Does Big Data Spell the End of Business Intelligence As We Know It?
Just as manual labourers were replaced by the machines of industry in the 19th century so certain IT roles will be swept away by cloud computing.That's the argument put forward by Gartner research director Gregor Petri - who believes that many roles managing IT infrastructure will all but disappear.Manual management of IT infrastructure - for instance provisioning additional storage, servers or network capacity for a particular application - will increasingly be automated as software layers in the cloud automatically divert IT resources to where they are needed, he said."It is very much like industrialisation," he said."Take the very old example given by Adam Smith of the pin makers who used to take a day to make four pins, then a factory is built that can make 10,000 pins in an hour. "That is what cloud computing is making possible: you can carry out these computing tasks on an industrial scale."Petri said that just as people no longer make pins manually, so in general … [Read more...] about Cloud computing: What does it really mean for IT jobs?
"Big Data" is a catch phrase that has been bubbling up from the high performance computing niche of the IT market. Increasingly suppliers of processing virtualization and storage virtualization software have begun to flog "Big Data" in their presentations. What, exactly, does this phrase mean? If one sits through the presentations from ten suppliers of technology, fifteen or so different definitions are likely to come forward. Each definition, of course, tends to support the need for that supplier's products and services. Imagine that. In simplest terms, the phrase refers to the tools, processes and procedures allowing an organization to create, manipulate, and manage very large data sets and storage facilities. Does this mean terabytes, petabytes or even larger collections of data? The answer offered by these suppliers is "yes." They would go on to say, "you need our product to manage and make best use of that mass of data." Just thinking about the problems created by the … [Read more...] about What is “Big Data?”